On choosing a tool, I prefer a different approach, i.e. select an integrated platform as early as possible. The layered approach - add tools as & when needed - while coming naturally, in my experience leads to fragmented architectures. One team might choose to add tool X (say W&B for experimenting), while another team chooses tool Y (say MLflow). I have even seen the same teams using different tools for the same purpose, but in different projects.
This approach, while seemingly efficient and flexible, leads to an overall system architecture that is ineffective, hard to manage and almost impossible to consolidate. Ultimately it ends up binding a lot of engineering capacity.
Disclaimer: I may be biased because I market an integrated platform that I have developed to solve this very problem.
I can definitely empathise with that perspective too. Different teams pick different tools all across the organisation and then you have to try to unpick them, decommissioning some and integrating others as you go. That's certainly one argument in favour of landing on a toolset early, and perhaps that would be sensible if you have (a) a really solid understanding of what you need from a toolset; and (b) the authority to make your choice the standard.
I think part of why there's such a proliferation of tools though is that teams choose to commit to a tool too early, unaware of what the broader governance requirements actually are, unaware of (or simply with no regard for) the tools that others have already chosen, and even unaware of what their own real needs are.
But you're right in pointing out that double-edged sword. Thanks
Thank you. Well written, on point, great read.
On choosing a tool, I prefer a different approach, i.e. select an integrated platform as early as possible. The layered approach - add tools as & when needed - while coming naturally, in my experience leads to fragmented architectures. One team might choose to add tool X (say W&B for experimenting), while another team chooses tool Y (say MLflow). I have even seen the same teams using different tools for the same purpose, but in different projects.
This approach, while seemingly efficient and flexible, leads to an overall system architecture that is ineffective, hard to manage and almost impossible to consolidate. Ultimately it ends up binding a lot of engineering capacity.
Disclaimer: I may be biased because I market an integrated platform that I have developed to solve this very problem.
I can definitely empathise with that perspective too. Different teams pick different tools all across the organisation and then you have to try to unpick them, decommissioning some and integrating others as you go. That's certainly one argument in favour of landing on a toolset early, and perhaps that would be sensible if you have (a) a really solid understanding of what you need from a toolset; and (b) the authority to make your choice the standard.
I think part of why there's such a proliferation of tools though is that teams choose to commit to a tool too early, unaware of what the broader governance requirements actually are, unaware of (or simply with no regard for) the tools that others have already chosen, and even unaware of what their own real needs are.
But you're right in pointing out that double-edged sword. Thanks