I started my career in software engineering about 15 years ago for an advertising agency. We had to do things quickly usually last second requests from clients. This lead to us never discussing the best approaches, things just needed to work, look pretty, and be done on time. It also taught me a valuable lesson that I'm sure many won't agree with me but I will try to explain my point of view. The lesson was no one cares about best practices if no one ever reuses that code.
Fast forward 15 years and I’m now working as Engineering Manager for a few years and I constantly see people over-engineering the solution to the point where they themselves don't know how to use their code in the next project. And this brings me here to discuss the notion of over architecture.
Throughout the years I started to be interested in the business side of things as much as the software side of things. This gave me some interesting insights into the minds of both sides of the development process. The decisions that management makes that don't make any sense to developers and the demands that developers request from management that seem completely ridiculous from the business side of things. The obvious scenario is business wants engineers to reuse as much code as they can from a previous project and basically focus on skinning the project to the new one. The developers will always come up with excuses that new requirements don't fit in current architecture and modifications need to be done on the foundational level of the project. And here we come to the core of the issue. If every time you get to “reuse” your project for a new project why do you need to upgrade the foundation or the architecture? and the answer is very simple in my opinion we cannot predict the future so why do we try to make software that takes care of every possible scenario?
Here is a very simple parallel; when a building is built the architect builds it based on the specs of the request x amount of dwellings, y amount of commercial space, etc… So when one of the living areas is rented out to a burger shop the burger shop will modify what is needed within the space that it is rented without consideration of what or who will live in that space in the future. Why don't we take a similar approach to software? The software architecture is the foundation of the type of projects it will support but it doesn't need to be more than that. There is no reason to worry about “all” types of add-ons that can possibly be added later on if the current project doesn't require them.
My recommendation to software engineers is to use the architecture that fits the project they are currently on, not the project that it can become in the future.
And I fully understand there are companies like FBs, googles, and amazons that do have millions of users and scenarios to cover so obviously their projects will require more robust architectures to be created, but the majority of programmers don't actually work for those companies, most people work for smaller companies that have a product or act as a professional service and this is where I recommend to keep it simple.
The lesson was no one cares about best practices if no one ever reuses that code.