18 Month Software Project Retrospective
This retrospective reflects my observations about technology and human matters as a result of working on a complex 2 year software development project in a geographically dispersed team. Key Successes Technical Achievements Process Improvements Project Management Challenges Individuals and Interactions over Processes and Tools Working Software over Comprehensive Documentation Customer Collaboration over Contract Negotiation Responding to Change over Following a Plan Challenges and Learnings Technical Challenges Team and Communication Architecture Decisions What Worked Well Technology Stack Team Dynamics Recommendations for Future Projects Personal Growth Lessons for Future Projects Key Successes Technical Achievements Successfully built a big data horizontally scalable ingestion system using Kubernetes and leaned into cloud native approaches early on Established heavy use of Python type hints early on, which improved code quality and editor aid Evangelised Elasticsearch early in the design phase: Led the adoption of Elasticsearch for read workloads, in the face of aprehension and inexperience in the broader team Implemented and tuned sophisticated text analysis pipelines Optimised search with ngram tokenizers, stemming, and asciifolds Designed efficient denormalised document structures and indexing strategies Lesson learned how important it is to make the the most appropriate data storage and management choices, make or break analytic solutions such as the one we collectively built. What consistency guarantees do are required? How fast? How are you going to calculate aggregations? What kind of read or write workloads need to be handled? Can these be separated and tackled as different problems? Elasticsearch is a HUGE reason why we were successful Created flexible hierarchy layering design, allowing differing customers to stamp the data with their own ways of doing things. Integrated OpenTelemetry for comprehensive observability Developed optimistic locking scheme and deep linking capabilities Automated deployments and quality verification with a gigantic test suite investment (unit and integration), linting, autoformatting, all orchestrated with a Makefile frontend and Bamboo CI pipeline The team embraced containers heavily from day 1. From running local vendor infra containers (redis, mongo, elasticsearch, etc) to running repeatable build workloads. Process Improvements Adopted Make for development automation, significantly boosting productivity Leveraged code generation effectively for complex scenarios, an ever powerful technique Implemented comprehensive integration testing with containerization Successfully broke down the system into functional components early Established well-defined data schemas upfront, which provided stability Project Management Challenges Individuals and Interactions over Processes and Tools Team structure and collaboration issues: ...