18 Month Software Project Retrospective

This retrospective reflects my observations about technology and human matters as a result of working on a complex 2 year software development project in a geographically dispersed team. Key Successes Technical Achievements Process Improvements Project Management Challenges Individuals and Interactions over Processes and Tools Working Software over Comprehensive Documentation Customer Collaboration over Contract Negotiation Responding to Change over Following a Plan Challenges and Learnings Technical Challenges Team and Communication Architecture Decisions What Worked Well Technology Stack Team Dynamics Recommendations for Future Projects Personal Growth Lessons for Future Projects Key Successes Technical Achievements Successfully built a big data horizontally scalable ingestion system using Kubernetes and leaned into cloud native approaches early on Established heavy use of Python type hints early on, which improved code quality and editor aid Evangelised Elasticsearch early in the design phase: Led the adoption of Elasticsearch for read workloads, in the face of aprehension and inexperience in the broader team Implemented and tuned sophisticated text analysis pipelines Optimised search with ngram tokenizers, stemming, and asciifolds Designed efficient denormalised document structures and indexing strategies Lesson learned how important it is to make the the most appropriate data storage and management choices, make or break analytic solutions such as the one we collectively built. What consistency guarantees do are required? How fast? How are you going to calculate aggregations? What kind of read or write workloads need to be handled? Can these be separated and tackled as different problems? Elasticsearch is a HUGE reason why we were successful Created flexible hierarchy layering design, allowing differing customers to stamp the data with their own ways of doing things. Integrated OpenTelemetry for comprehensive observability Developed optimistic locking scheme and deep linking capabilities Automated deployments and quality verification with a gigantic test suite investment (unit and integration), linting, autoformatting, all orchestrated with a Makefile frontend and Bamboo CI pipeline The team embraced containers heavily from day 1. From running local vendor infra containers (redis, mongo, elasticsearch, etc) to running repeatable build workloads. Process Improvements Adopted Make for development automation, significantly boosting productivity Leveraged code generation effectively for complex scenarios, an ever powerful technique Implemented comprehensive integration testing with containerization Successfully broke down the system into functional components early Established well-defined data schemas upfront, which provided stability Project Management Challenges Individuals and Interactions over Processes and Tools Team structure and collaboration issues: ...

June 20, 2025 · 7 min

Clean Architecture

Domain centric architectures, like clean architecture, have inner architectural cores that model the domain. Dependency inversion is king, with inner layers defining abstractions and interfaces and outer layers implementing them. Clean architecture is a good fit when aligning to Domain Driven Design (DDD), dealing with complex business logic, high testability is desirable and/or working in a large team, as the architecture can enforce design policies. Glossary Guiding Principles Clean Architecture Layers Domain layer Entities Value Objects Domain Events Domain Services Interfaces Results and Exceptions Application layer: The Use Case Orchestrator Application Layer Key Responsibilities Use Case Orchestration Higher Order Business Logic Cross Cutting Concerns Exception Translation & Handling Dependency Injection Hub What the Application Layer Does NOT Do Example Application Service Dependency Injection and MediatR Bootstrapping CQRS Abstractions Handling Domain Events Cross Cutting Concerns with MediatR Pipelines Logging Pipeline Validation Pipeline with FluentValidation Infrastructure layer Infrastructure Layer Key Responsibilities Data Persistence and Access External Service Integration Cross Cutting Concerns Implementation Event Handling Infrastructure What the Infrastructure Layer Does NOT Do Example Concrete Provider for IDateTimeProvider EF Core Setup Integrating Domain Entities with EF Core Publishing Domain Events in the Unit of Work Handling Race Conditions with Optimistic Concurrency Presentation layer Presentation Layer Key Responsibilities Input Handling and Validation Request Translation Response Formatting Authentication and Authorization Error Handling and Translation Dependency Injection Configuration What the Presentation Layer Does NOT Do Business Logic Data Access Complex Validation State Management Cross-Cutting Concerns Implementation API Controllers and Endpoints Seed Data and EF Migrations .NET Implementation Tips General .NET Tips Domain Layer .NET Tips Application Layer .NET Tips Infrastructure Layer .NET Tips Presentation Layer .NET Tips Bonus: Contemporary .NET gems Primary Constructors Switch Expressions Records Async Tips MediatR IRequest and IRequestHandler - Request/Response Publishing INotification and INotificationHandler - Pub/Sub Publishing MediatR.Contracts Package Visual Studio and Roslyn Code Quality Level Ups dotnet CLI Tips Glossary Term Definition Aggregate A cluster of domain objects that can be treated as a single unit. An aggregate has one aggregate root and enforces consistency boundaries. Aggregate Root The only member of an aggregate that outside objects are allowed to hold references to. It controls access to the aggregate’s internals. Anemic Domain Model An anti-pattern where domain objects contain little or no business logic, acting mainly as data containers with getters and setters. Application Service A service in the application layer that orchestrates domain objects and infrastructure to fulfill use cases. Bounded Context A central pattern in DDD that defines explicit boundaries within which a domain model is valid and consistent. Clean Architecture An architectural pattern that separates concerns into concentric layers, with dependencies pointing inward toward the domain. Command An object that represents a request to perform an action, often used in CQRS to separate write operations. Context Map A visual representation showing the relationships and integration patterns between different bounded contexts. CQRS Separating read and write operations into different models and potentially different databases. Dependency Inversion A principle stating that high-level modules should not depend on low-level modules; both should depend on abstractions. Domain The subject area or sphere of knowledge and activity around which the application logic revolves. Domain Event Something that happened in the domain that domain experts care about and that triggers side effects. Domain Model An object model of the domain that incorporates both behavior and data, representing the business concepts and rules. Domain Service A service that encapsulates domain logic that doesn’t naturally fit within a single entity or value object. Entity A domain object that has a distinct identity that runs through time and different states. Event Sourcing A pattern where state changes are stored as a sequence of events rather than just the current state. Hexagonal Architecture Also known as Ports and Adapters, isolates the core business logic from external concerns through well-defined interfaces. Infrastructure Layer The outermost layer containing technical details like databases, external APIs, and frameworks. Onion Architecture Similar to Clean Architecture, organizing code in concentric layers with dependencies pointing inward. Port An interface that defines how the application core communicates with external systems (part of Hexagonal Architecture). Query In CQRS, a request for data that doesn’t change system state, optimized for reading operations. Repository A pattern that encapsulates the logic needed to access data sources, centralising common data access functionality. Rich Domain Model A domain model where business logic is encapsulated within domain objects rather than external services. Saga A pattern for managing long-running business processes that span multiple aggregates or bounded contexts. Specification Pattern A pattern used to encapsulate business rules and criteria that can be combined and reused. Ubiquitous Language A common language shared by developers and domain experts within a bounded context. Use Case A specific way the system is used by actors to achieve a goal, often implemented as application services. Value Object An object that describes characteristics or attributes but has no conceptual identity. Guiding Principles High level qualities that a good software architecture should (and enforce) strive for; maintainability, testability and loose coupling. ...

May 29, 2025 · 42 min