Kibana

Intro Install Package (yum or deb) X-Pack Configuration Verify Index Patterns Intro Elastics node.js web frontend in the stack, and (by default) runs on port 5601. It’s wise to install Kibana on its own infrastructure (i.e. isolated from the Elasticsearch cluster). The node process is light (compared to the JVM anyway) consuming hundreds of MB. Install Package (yum or deb) While available as a tarball, the nicest option is to go with a package, takes care of plumbing such as systemd, and general system integration such as /etc/kibana for configuration, logs and data files. ...

January 24, 2019 · 2 min

PKI

Some of my (and others) notes of managing PKI with the excellent openssl. Its simple and just works. To get going will create a root CA (Certificate Authority) and an intermediate signing CA. Using the CA’s will issue three keypairs; one for email protection, one for TLS, and one for digital signatures. The digital signature keypair will be presented in the form of a CSR (Certificate Signing Request), as if generated by a third party that would like a keypair, signed by our CA hierarchy. ...

January 8, 2018 · 14 min

Computer Heros

Some heros in the world of computer science. Founding Fathers (pre 1960) George Boole (1815 - 1864) Boolean algegra. Papers The Mathematical Analysis of Logic (1847), and An Investigation of the Laws of Thought (1854). Friedrich Ludwig Gottlob Frege (1848 - 1925) Analytic philosophy. Developed modern logic in the Begriffsschrift and work in the foundations of mathematics. Books The Foundations of Arithmetic, On Sense and Reference and The Thought. Alan Turing (1912 - 1954) The father of theoretical computer science. Formalised the concepts of algorithm and computation through the notion of a Universal Machine (i.e Turing Machine) in his seminal paper On Computable Numbers, with an Application to the Entscheidungsproblem. ...

April 27, 2017 · 2 min

DIY Computer Part 5 Machine Architecture

A continuation of my participation in the amazing Nand2Tetris course, by Noam Nisan and Shimon Schocken, now running on Coursera. Related posts: [DIY Computer Part 1 The NAND Gate]({% post_url 2016-03-06-diy-computer-nands %}) [DIY Computer Part 2 The ALU]({% post_url 2016-03-12-diy-computer-alu %}) [DIY Computer Part 3 Memory]({% post_url 2016-03-12-diy-computer-alu %}) [DIY Computer Part 4 Machine Language]({% post_url 2016-05-23-diy-computer-ml %}) Now armed with the necessary elementary logic gates and chips, are now ready to assemble a general purpose computer. Using a finite amount of hardware, it will be capable of performing practically an infinite number of tasks (e.g. games, word processing, networking). This remarkable flexibility, the fruit of the profound invention of several mathematicians in the 1930’s, is known as the stored program. ...

April 17, 2017 · 6 min

Nerd Gems 💎

This is a list of valuable (to me) developer resources that I’ve managed to stumble across (books, courses, friends and fellow programmers, hacker news, lobste.rs, university). Architecture AI and ML Awesome C Cloud Containers Cheat sheets Culture Databases Development Diagramming dotfiles Git Golang gRPC Hardware Humanities Jobs Kubernetes Languages Linux Message queues Mongo Monitoring Networking Open source Python Rust Security Shell systemd Text wrangling Talks Terminal Testing Text, Encoding and Serialization TypeScript Web Architecture .NET Microservices: Architecture for Containerized .NET Applications a fantastic resource for working with the modern .NET stack (post 2022) Communicating Sequential Processes Tony Hoare’s seminal 1977 paper on concurrency and CSP Why Segment Went Back to a Monolith microservices come with serious tradeoffs All software sucks complexity is the bane of all software, simplicity is the most important quality Designing Actor-Based Software with Hugh McKee an approach to building scalable software systems Queueing: An interactive study of queueing strategies an interactive journey to understand common queueing strategies for handling HTTP requests. AI and ML Andrej Karpathy on The spelled-out intro to neural networks and backpropagation: building micrograd a 2.5 hour step-by-step spelled-out explanation of backpropagation and training of neural networks. It only assumes basic knowledge of Python and a vague recollection of calculus from high school. A Beginner’s Guide to Vector Embeddings Awesome awesome-baremetal awesome-kubernetes learn-anything/books C Easy Makefile a Makefile boilerplate to hit the ground running Handmade Hero an educational series by Casey Muratori that teaches low-level game programming techniques by example Eskil Steenberg on How I program in C Cloud mcm Minimal Configuration Manager Packer a tool for building images for cloud platforms, virtual machines, containers and more from a single source configuration. CloudBoost a complete serverless platform for your app. The Google Cloud Developer’s Cheat Sheet every product in the Google Cloud family described in under 4 words Ask HN: Is Your Company Sticking to On-Premise Servers? Why? Using AWS CodeBuild to Execute Administrative Tasks What Is Amazon Resource Name (ARN)? more to the humble ARN than you think arn:partition:service:region:account-id:resource Containers OKD: The Community Distribution of Kubernetes that powers OpenShift The Docker Handbook 25 Basic Docker Commands for Beginners Setting the Record Straight: containers vs. Zones vs. Jails vs. VMs Docker Security Best Practices tools and methods to help secure Docker Kubernetes Workshop tons of details for getting started 10 Most Common Mistakes When Using Kubernetes lens kube IDE kubeseal how to safely store secrets in git if you want to use them in k8s Container Technologies at Coinbase great history on how the industry got to containers, an why kubernetes isn’t used A Practical Introduction to Container Security Webtop full desktop environments in officially supported flavors accessible via any modern web browser Cheat sheets Linux Commands - A practical reference an amazing cheat sheet, quick reference The Ultimate List of SANS Cheat Sheets when it comes to quality cyber-security training and certs SANS is world leading. They have an amazing collection of thoughtful and useful cheat sheets from topics such as Writing Tips for IT Professionals, Windows to Unix Cheat Sheet, to using pieces of software such as nmap, netcat, burb. Its a treasure trove! Lenny Zeltser’s IT and Information Security Cheat Sheets speaking of thoughtful cheat sheets, lots of wisdom here Culture Why we’re leaving the cloud You Are Not Google if you’re using a technology that originated at a large company, but your use case is very different, it’s unlikely that you arrived there deliberately; no, it’s more likely you got there through a ritualistic belief that imitating the giants would bring the same riches. GitLab’s Guide to All-Remote the remote manifesto, tips and tricks and remote resources Why we at $FAMOUS_COMPANY Switched to $HYPED_TECHNOLOGY Habbits of High-Functioning Software Teams characteristics and habits of the highest-performing dev teams Databases Things I Wished More Developers Knew About Databases SQL Coding Standards PostgreSQL Course: A Curious Moon learn PostgreSQL the way the pros do: on the job and under pressure. You’ll assume the role of interim DBA at aerospace startup Red:4, exploring data from the Cassini mission! Development What To Code inspiration and ideas Why the developers who use Rust love it so much Diagramming Excalidraw beautiful web based diagrams PlantText PlantUML (text) based diagram generator Taking ASCII Drawings Seriously: How Programmers Diagram Code dotfiles HexDSL LukeSmithxyz uoou Git Better Git configuration links and resources on configuring & using git Automate Repetitive Tasks with Custom git Commands how to write custom git commands Golang Everyday Golang LearnGo: A Large Collection of Go Examples, Exercises, and Quizzes Writing Go CLIs With Just Enough Architecture Getting Hands-On with io_uring from Go Diving Into Go by Building a CLI Application Immutability Patterns in Go Writing An Interpreter In Go gRPC gRPC - Best Practices Hardware Backblaze hard drive stats Build an 8-bit CPU by Ben Eater a programmable 8-bit computer from scratch on breadboards using only simple logic gates nand2tetris a distilled version of the book The Elements of Computing Systems, By Noam Nisan and Shimon Schocken (MIT Press), contains all the project materials and tools necessary for building a general-purpose computer system and a modern software hierarchy from the ground up Humanities The Chomsky List A definitive guide to Noam Chomsky: 10 books to get you started RATM reading list Jobs Inspired corp Kubernetes 7 Mind-Blowing Kubernetes Hacks The guide to kubectl I never had The Pros of On-Prem Kubernetes with Justin Garrison Languages Crafting Interpreters by Robert Nystrom Ever wanted to make your own programming language or wondered how they are designed and built? If so, this book is for you. Linux Linux Commands - A practical reference an amazing cheat sheet, quick reference 16 Linux server monitoring commands you really need to know Best 15 Unix Command Line Tools An In-Depth Guide to iptables covers pretty much every angle of iptables, from basic rules to NAT’ing to protocols and interfaces. mdadm Cheat Sheet practical commands when running software raid on Linux Async IO on Linux: select, poll, and epoll thorough write-up on ‘select’, ‘poll’ and ’epoll’ system calls, and how to measure them. The first 5 things to do when your Linux server keels over including hardware troubleshooting, checking the running state of applications How io_uring and eBPF Will Revolutionize Programming in Linux well explained history of Linux syscalls and their limitations, and how io_uring is a game changer by allowing async I/O via a pub/sub model bashtop gamified TUI resource monitor that shows usage and stats for processor, memory, disks, network and processes Time on Unix how time and localization works on Unix Tmux for mere mortals good defaults, modifying the keybindings to boost usability Tips for cleaning up a Linux server low hanging disk space fruit, like removing old kernels, pruning unused Docker space, clearing logs Shell productivity tips and tricks faster command line tips Message queues Postgres Message Queue - PGMQ lightweight message queue, like AWS SQS and RSMQ but on Postgres Mongo Quick reference cards Aggregation pipeline quick reference Monitoring Zabbix whatfiles logs the files programs CRUD, also traces new processes logtop reads stdin, can sort on any field and is updated in realtime Networking PacketLife Cheat Sheets The Packet Pioneer Chris Greer on TCP Fundamentals Part 1 TCP/IP Explained with Wireshark 59 Linux Networking commands and scripts the ultimate network tools goto list. Introduction to tcpdump and wireshark hping3 send arbitary TCP/IP packets to network hosts Setting up a Linux mail server linker∙d dynamic linker for microservices, taking care of the communication work needed to interact with distributed services, including routing, load balancing, and retrying. Manually Throttle the Bandwidth of a Linux Network Interface introduction to the tc tool for bandwidth shaping. connbeat agent that monitors TCP connection metadata and ships the data to Kafka or Elasticsearch, or an HTTP endpoint The Ultimate PCAP all protocols in a single PCAP What Every Developer Should Know About TCP SSH Tips & Tricks 2FA, securely forwarding agents, quitting from stuck sessions and using mosh or tmux High Availability Load Balancers with Maglev CloudFlare on their load balancing stack, BGP, Maglev connection scheduling, IPVS, UDP encapsulation for faster delivery Networking for Game Programmers: UDP vs TCP Open source Google Open Source 2000+ OSS projects managed by Google NSA on GitHub Python Python Design Patterns Inside the Python Virtual Machine Full Speed Python from Superior School of Technology of Setúbal Intermediate Python Ruff: Internals of a Rust-backed Python linter-formatter - Part 1 A Guide to Python’s Weak References Using weakref Module A Complete Guide to Pytest Fixtures Rust 100 Exercises To Learn Rust teaches Rust’s core concepts, one exercise at a time. You’ll learn about Rust’s syntax, its type system, its standard library, and its ecosystem. Security OST2.FYI OpenSecurityTraining2’s mission is to provide the world’s deepest and best cybersecurity training. That our classes are free is just a bonus! The Ultimate List of SANS Cheat Sheets when it comes to quality cyber-security training and certs SANS is world leading. They have an amazing collection of thoughtful and useful cheat sheets from topics such as Writing Tips for IT Professionals, Windows to Unix Cheat Sheet, to using pieces of software such as nmap, netcat, burb. Its a treasure trove! Lenny Zeltser’s IT and Information Security Cheat Sheets speaking of thoughtful cheat sheets, lots of wisdom here Linux reverse engineering 101 collection of resources for linux reverse engineering. Explain like I’m 5: Kerberos OAuth 2.0 Security Best Current Practices SSHHeatmap script that generates a heatmap of IP’s that made failed SSH login attempts using /var/log/auth.log psst Paper-based Secret Sharing Technique Shell Byobu multiplexer, enhanced profiles, convenient keybindings, configuration utilities, and toggle-able system status notifications for screen and tmux Makeself a self-extracting archiving tool for Unix systems, in 100% shell script 5 Types Of ZSH Aliases You Should Know alias suffixes & global aliases, plus other neat tricks Bash aliases you can’t live without systemd Why I Prefer systemd Timers Over Cron journal-triggerd runs trigger on systemd’s journal messages How to automatically execute shell script at startup boot on systemd Text wrangling CyberChef the ultimate open-source (by GCHQ) text wrangler you’ll ever need, life changing desed beautiful TUI that provides users with comfortable interface and practical debugger, used to step through complex sed scripts sed One Liners huge collection of useful sed examples xsv CLI for indexing, slicing, analyzing, splitting and joining CSV files Talks Rich Hickey on Simple Made Easy Mike Acton on Data-orientated Design Jonathan Blow on Programming Aesthetics learned from making independent games Eskil Steenberg on How I program in C Rich Hickey on Hammock Driven Development Brian Will on Why OOP is Bad Abner Coimbre on What Programming is Never About Scott Meyers on CPU Caches and Why You Care Jeff and Casey Show on The Evils of Non-native Programming Jeff and Casey’s Guide to Becoming a Bigger Programmer Hadi Hariri on The Silver Bullet Syndrome Bryan Cantrill on Fork Yeah! The Rise and Development if illumos Rob Pike on Concurrency Is Not Parallelism James Mickens on JavaScript Liz Rice on Containers From Scratch James Mickens on Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible? Terminal Terminal Text Effects visual effects applied to text in the terminal Testing Smocker simple HTTP mock server, uses YAML to define mocks and responses MockServer for any system you integrate with via HTTP or HTTPS MockServer can be used as: a mock configured to return specific responses for different requests, a proxy recording and optionally modifying requests and responses or as both a proxy for some requests and a mock for other requests at the same time Text, Encoding and Serialization The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!) Illustrated jq tutorial jq is a lightweight and flexible command-line JSON processor Cap’n Proto Google Protocol Buffers TypeScript The Consise TypeScript Book Web HTML5 UP makes spiffy HTML5 site templates that are HTML5 + CSS3, customizable and 100% free under the Creative Commons How I built a modern website in 2021 Certbot automatically use Let’s Encrypt certificates Ask HN: Is There Still a Place for Native Desktop Apps? topngx parse and aggregrate statistics from NGINX access logs

April 2, 2017 · 10 min

Shay Banon on Innovation

Some great wisdom imparted by Shay Banon (creator of Elasticsearch). Be bored. To promote innovation. Open source and building a critical mass. Be committed. You need to care for it now; IRC, forums, negative feedback, bugs. The benevolent dicator model (BDM) versus the democracy with a leader model. Be distributed. Choose a useful (and normally hard) problem to solve. Best way to learn is to actually build something. Double down on passion. Recruit passionate people (e.g. story about the guy who paid for Elastic training off his own back). Videos are for collaboration. Simple things, should be simple. Not easy though. Hierarchy matters. Humans can’t make decisions in groups. IT is obsessed with whats next. You only need to look backwards for answers.

November 22, 2016 · 1 min

DIY Computer Part 4 Machine Language

A continuation of my participation in the amazing Nand2Tetris course, by Noam Nisan and Shimon Schocken, now running on Coursera. In this course you will build a modern computer system, from the ground up. We’ll take you from constructing elementary logic gates all the way through creating a fully functioning general purpose computer. In the process, you will learn how really computers work, and how they are designed. If interested, see prior posts: ...

May 23, 2016 · 7 min

DIY Computer Part 2 The ALU

A continuation of my participation in the amazing Nand2Tetris course, by Noam Nisan and Shimon Schocken, now running on Coursera. In this course you will build a modern computer system, from the ground up. We’ll take you from constructing elementary logic gates all the way through creating a fully functioning general purpose computer. In the process, you will learn how really computers work, and how they are designed. If interested, see prior related post [DIY Computer Part 1 The NAND Gate]({% post_url 2016-03-06-diy-computer-nands %}). ...

March 12, 2016 · 5 min

DIY Computer Part 1 The NAND Gate

I love computers. When I recently discovered that the Nand2Tetris guys, Noam Nisan and Shimon Schocken, had just packaged their famous course into a neat 12 week Coursera package, I couldn’t resist studying it any longer. In this course you will build a modern computer system, from the ground up. We’ll take you from constructing elementary logic gates all the way through creating a fully functioning general purpose computer. In the process, you will learn how really computers work, and how they are designed. ...

March 6, 2016 · 7 min

Nerd Conferences

One of the best ways to evolve in the software industry is to interact with other developers and those technically minded. Resources such as user groups, Twitter and conferences are useful for this. It’s disturbingly easy to develop reality distortion or tunnel vision with the projects and technology stacks you spend a majority of your time with. Conferences on my radar: January linux.conf.au, Geelong Febuary APIdays, Sydney ...

April 11, 2015 · 1 min