Software

Domain centric architectures, like clean architecture, have inner architectural cores that model the domain. Dependency inversion is king, with inner layers defining abstractions and interfaces and outer layers implementing them. Clean architecture is a good fit when aligning to Domain Driven Design (DDD), dealing with complex business logic, high testability is desirable and/or working in a large team, as the architecture can enforce design policies. Guiding Principles Clean Architecture Layers Domain layer Entities Value Objects Domain Events Domain Services Interfaces Results and Exceptions Application layer: The Use Case Orchestrator Application Layer Key Responsibilities Use Case Orchestration Higher Order Business Logic Cross Cutting Concerns Exception Translation & Handling Dependency Injection Hub What the Application Layer Does NOT Do Example Application Service Dependency Injection and MediatR Bootstrapping CQRS Abstractions Handling Domain Events Cross Cutting Concerns with MediatR Pipelines Logging Pipeline Validation Pipeline with FluentValidation Infrastructure layer Infrastructure Layer Key Responsibilities Data Persistence and Access External Service Integration Cross Cutting Concerns Implementation Event Handling Infrastructure What the Infrastructure Layer Does NOT Do Example Concrete Provider for IDateTimeProvider EF Core Setup Integrating Domain Entities with EF Core Publishing Domain Events in the Unit of Work Handling Race Conditions with Optimistic Concurrency Distributed Cache Service Presentation layer Presentation Layer Key Responsibilities What the Presentation Layer Does NOT Do API Controllers and Endpoints Seed Data and EF Migrations Authentication (authn) with Keycloak Authorization (authz) Role-based Authorization Permission-based (Policy) Authorization Resource-based Authorization Structured Logging Health Checks .NET Implementation Tips General .NET Tips Domain Layer .NET Tips Application Layer .NET Tips Infrastructure Layer .NET Tips Presentation Layer .NET Tips Structured Logging Serilog Serilog and Seq Setup Guide Outbox Pattern The Problem The Solution Key Benefits Outbox .NET Implementation Outbox Message Definition Transactionally Publish Domain Events as Outbox Messages Background Worker Job with Quartz.NET Hookup Dependency Injection ASP.NET Core Minimal APIs Controller to Minimal API Conversion Cookbook Centralising Route Opinions with Route Groups Testing Domain Layer Unit Testing Application Layer Unit Testing Mocking with NSubstitute Application Layer Integration Testing with TestContainers Troubleshooting Accessing Internal Symbols Bonus: Contemporary .NET gems Primary Constructors Switch Expressions Records Async Tips MediatR IRequest and IRequestHandler - Request/Response Publishing INotification and INotificationHandler - Pub/Sub Publishing MediatR.Contracts Package Visual Studio and Roslyn Code Quality Level Ups dotnet CLI Tips Guiding Principles High level qualities that a good software architecture should (and enforce) strive for; maintainability, testability and loose coupling. ...

What worked, what didn’t, and concrete recommendations for future projects. My in the trenches opinions, observations and reflections of building a complex distributed software system across multiple geographically dispersed teams. Key Successes Technical Achievements Process Improvements Project Management Challenges Forgotten Agile Principle 1: Individuals and Interactions over Processes and Tools Forgotten Agile Principle 2: Working Software over Comprehensive Documentation Forgotten Agile Principle 3: Customer Collaboration over Contract Negotiation Forgotten Agile Principle 4: Responding to Change over Following a Plan Challenges and Learnings Technical Challenges Team and Communication Architecture Decisions What Worked Well Technology Stack Team Dynamics Recommendations for Future Projects Personal Growth Lessons for Future Projects Key Successes Technical Achievements Successfully built a big data horizontally scalable ingestion system using Kubernetes and leaned into cloud native approaches early on Established heavy use of Python type hints early on, which improved code quality and editor aid Evangelised Elasticsearch early in the design phase: Led the adoption of Elasticsearch for read workloads, in the face of aprehension and inexperience in the broader team Implemented and tuned sophisticated text analysis pipelines Optimised search with ngram tokenizers, stemming, and asciifolds Designed efficient denormalised document structures and indexing strategies Lesson learned how important it is to make the the most appropriate data storage and management choices, make or break analytic solutions such as the one we collectively built. What consistency guarantees do are required? How fast? How are you going to calculate aggregations? What kind of read or write workloads need to be handled? Can these be separated and tackled as different problems? Elasticsearch is a HUGE reason why we were successful Created flexible hierarchy layering design, allowing differing customers to stamp the data with their own ways of doing things. Integrated OpenTelemetry for comprehensive observability Developed optimistic locking scheme and deep linking capabilities Automated deployments and quality verification with a gigantic test suite investment (unit and integration), linting, autoformatting, all orchestrated with a Makefile frontend and Bamboo CI pipeline The team embraced containers heavily from day 1. From running local vendor infra containers (redis, mongo, elasticsearch, etc) to running repeatable build workloads. Process Improvements Adopted Make for development automation, significantly boosting productivity Leveraged code generation effectively for complex scenarios, an ever powerful technique Implemented comprehensive integration testing with containerization Successfully broke down the system into functional components early Established well-defined data schemas upfront, which provided stability Project Management Challenges This project mangaged to defy every Agile Manifesto Principle. ...