Posts

PostgreSQL (postgres or pgsql) is a powerful open source relational database known for reliability, extensibility, and standards compliance. It features: Advanced SQL support (window functions, CTEs, JSON, full-text search) ACID compliance and strong transactional integrity Rich indexing (B-tree, GIN, GiST, BRIN, hash, SP-GiST) Extensible with custom types, operators, and functions MVCC for high concurrency and performance Robust security, authentication, and role management Active community, frequent releases, and excellent documentation Ideal for everything from small apps to large-scale, mission-critical systems. ...

Recently I’ve had the opportunity to dig into Apache Spark, thanks to some training from Brian Bloechle from Cloudera. What is spark? Fast, flexible, and developer friendly, Apache Spark is the leading platform for large scale SQL, batch processing, stream processing, and machine learning. Java, Scala, Python and R are first class citizens when its comes to consuming the various Spark API’s. I’ll cover PySpark in more detail. Spark is an agnostic processing engine, that can target a number of cluster managers including Spark Standalone, Hadoop’s YARN, Apache Mesos and Kubernetes. In the context of Spark, some useful surrounding ecosystem to be aware of: ...

Installation on Linux is well documented. wget https://www.scootersoftware.com/bcompare-4.2.4.22795.x86_64.rpm su rpm --import https://www.scootersoftware.com/RPM-GPG-KEY-scootersoftware yum install bcompare-4.2.4.22795.x86_64.rpm On my Fedora 27 box, I found the launcher bash /usr/bin/bcompare exits with a return code of 1, if the linker cannot resolve all dependencies. BC_LIB=/usr/lib64/beyondcompare export BC_LIB EXEC=$BC_LIB/BCompare #check to see if we have all of the shared libraries. CHECK=`ldd $EXEC | grep "not found" | wc -l` if [ "$CHECK" -ne "0" ]; then echo Some Shared Libraries were not found ldd $EXEC exit 1 fi Dumping out the shared libs BCompare depends on: ...

Hey there, these are some of my notes about IRC in general, and the CLI client irssi. IRC Primer Internet Relay Chat (IRC) is an open protocol that uses TCP and optionally TLS. IRC was born in August 1988, by Jarkko Oikarinen at the University of Oulu in Finland. Years later, in 1993, was standardised as RFC 1459. Assigned port 194/TCP by IANA, however in practice most use 6667/TCP, and nearby ports (6660-6669, 7000). ...

I’m a convert. This conference was incredible, with passionate presenters, excellent content with an amazing inclusive and friendly community vibe. Most sessions are available on YouTube. Things I am inspired to do post LCA 2018: Setup ctags with Vim. Think ‘go to definition’ for Vim, with support for 40+ langs. Specifically Exuberant Ctags and unite-tag. Play with more kernel syscalls (like ptrace). Signal processing and logic analyser lab and kit. More notes. Resume the nand2tetris journey, part 2. Use IRC more. Some CLI options Irssi BitchX WeeChat. And keybase. Imagine a Slack for the whole world, except end-to-end encrypted across all your devices. Or a Team Dropbox where the server can’t leak your files or be hacked. Always support and subscribe to lwn.net. Look into supporting the Software Conservancy Foundation (SCF) Pick up some new langs; go and rust seem to be current shiny. I met a number of kernel developers. Find out a bit more about what they do through the offical MAINTAINERS file. Yubikey or Nitrokey, start using. Open source physical USB device to enable secure encryption and signing of data. The secret keys are always stored inside the Nitrokey which protects against malware (such as computer viruses) and attackers. A user-chosen PIN and a tamper-proof smart card protect the Nitrokey in case of loss and theft. Linux Device Drivers book now maintained by Jonathan Corbet (executive editor at lwn.net), start reading, and create drivers, the gateway drug to the kernel. Time to refresh handle, bio, and CV. Checkout TaskWarrior, a CLI for managing tasks. Consider using more compact unique IDs such as Twitters snowflake e.g. 12738165059 Use more state machines. Study The Architecture of Open Source Applications. Some of the sessions I got to attend. ...

Born in 2008, LXC (pronouced lexy) is a userspace interface for the Linux kernel containment features, that enables the creation and management of application containers. LXC leverages a number of kernel features to contain processes: Kernel namespaces (ipc, uts, mount, pid, network and user) Apparmor and SELinux profiles Seccomp policies Chroots (using pivot_root) Kernel capabilities CGroups (control groups) LXC containers are often considered as something in between a chroot and a full blown VM. The goal of LXC is to create an environment as close as possible to a standard Linux installation, without the need for a separate kernel. ...

Some of my (and others) notes of managing PKI with the excellent openssl. Its simple and just works. To get going will create a root CA (Certificate Authority) and an intermediate signing CA. Using the CA’s will issue three keypairs; one for email protection, one for TLS, and one for digital signatures. The digital signature keypair will be presented in the form of a CSR (Certificate Signing Request), as if generated by a third party that would like a keypair, signed by our CA hierarchy. ...

Ansible is a radically simple IT automation platform that makes your applications and systems easier to deploy. Avoid writing scripts or custom code to deploy and update your applications. Automate in a language that approaches plain English, using SSH, with no agents to install on remote systems. With Ansible and its amazing module ecosystem, you describe what needs to be accomplished (i.e. declarative), rather than describing how to accomplish each step (i.e. imperative). ...

When something becomes messy in the software industry, we have the habit of tucking the problems away behind abstractions. To me Vagrant is simply an abstraction layer for working with a range of hypervisors (libvirt, VirtualBox, VMWare, Hyper-V, Docker and more), in a reliable and consistent manner. By articulating VM’s as a simple text based (Ruby) configuration file, a Vagrantfile, achieve greater simplicity, portability across systems, and consistency which could theoretically be used with any virtual environment running on top of any other system. ...

Partitioning Two popular partition schemes are used in the wild, MBR and GPT. MBR MBR, or Master Boot Record, often associated with BIOS, was introduced in 1983 with IBM PC DOS 2.0, is a special boot sector located at the beginning of a drive. This sector contains a boot loader (e.g GRUB), and details about the logical partitions. MBR supports drives upto 2TiB, and up to 4 primary partitions. # fdisk /dev/vda Welcome to fdisk (util-linux 2.23.2). Changes will remain in memory only, until you decide to write them. Be careful before using the write command. Device does not contain a recognized partition table Building a new DOS disklabel with disk identifier 0x9228f9b7. Command (m for help): m Command action a toggle a bootable flag b edit bsd disklabel c toggle the dos compatibility flag d delete a partition g create a new empty GPT partition table G create an IRIX (SGI) partition table l list known partition types m print this menu n add a new partition o create a new empty DOS partition table p print the partition table q quit without saving changes s create a new empty Sun disklabel t change a partition's system id u change display/entry units v verify the partition table w write table to disk and exit x extra functionality (experts only) Command (m for help): n Partition type: p primary (0 primary, 0 extended, 4 free) e extended Select (default p): Using default response p Partition number (1-4, default 1): First sector (2048-2097151, default 2048): Using default value 2048 Last sector, +sectors or +size{K,M,G} (2048-2097151, default 2097151): +500M Partition 1 of type Linux and of size 500 MiB is set Command (m for help): l 0 Empty 24 NEC DOS 81 Minix / old Lin bf Solaris 1 FAT12 27 Hidden NTFS Win 82 Linux swap / So c1 DRDOS/sec (FAT- 2 XENIX root 39 Plan 9 83 Linux c4 DRDOS/sec (FAT- 3 XENIX usr 3c PartitionMagic 84 OS/2 hidden C: c6 DRDOS/sec (FAT- 4 FAT16 <32M 40 Venix 80286 85 Linux extended c7 Syrinx 5 Extended 41 PPC PReP Boot 86 NTFS volume set da Non-FS data 6 FAT16 42 SFS 87 NTFS volume set db CP/M / CTOS / . 7 HPFS/NTFS/exFAT 4d QNX4.x 88 Linux plaintext de Dell Utility 8 AIX 4e QNX4.x 2nd part 8e Linux LVM df BootIt 9 AIX bootable 4f QNX4.x 3rd part 93 Amoeba e1 DOS access a OS/2 Boot Manag 50 OnTrack DM 94 Amoeba BBT e3 DOS R/O b W95 FAT32 51 OnTrack DM6 Aux 9f BSD/OS e4 SpeedStor c W95 FAT32 (LBA) 52 CP/M a0 IBM Thinkpad hi eb BeOS fs e W95 FAT16 (LBA) 53 OnTrack DM6 Aux a5 FreeBSD ee GPT f W95 Ext'd (LBA) 54 OnTrackDM6 a6 OpenBSD ef EFI (FAT-12/16/ 10 OPUS 55 EZ-Drive a7 NeXTSTEP f0 Linux/PA-RISC b 11 Hidden FAT12 56 Golden Bow a8 Darwin UFS f1 SpeedStor 12 Compaq diagnost 5c Priam Edisk a9 NetBSD f4 SpeedStor 14 Hidden FAT16 <3 61 SpeedStor ab Darwin boot f2 DOS secondary 16 Hidden FAT16 63 GNU HURD or Sys af HFS / HFS+ fb VMware VMFS 17 Hidden HPFS/NTF 64 Novell Netware b7 BSDI fs fc VMware VMKCORE 18 AST SmartSleep 65 Novell Netware b8 BSDI swap fd Linux raid auto 1b Hidden W95 FAT3 70 DiskSecure Mult bb Boot Wizard hid fe LANstep 1c Hidden W95 FAT3 75 PC/IX be Solaris boot ff BBT 1e Hidden W95 FAT1 80 Old Minix Command (m for help): t Selected partition 1 Hex code (type L to list all codes): 83 Changed type of partition 'Linux' to 'Linux' Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. The partition is now available as a block device, below we now see /dev/vda1: ...