Welcome to The Neural Node, my blog about AI infrastructure and system architecture.

After being an enthusiast of AI for 12 years and having worked in the industry for 5 years, I thought it’s time to share my knowledge and thoughts about the space in a blog.

One thing that fascinates me is the possibility of building your own AI systems at home with the same stack of software tooling and infrastructure, big tech is using in their labs. And all that with commodity hardware from eBay and open source software.

————————————————————————————-

fascination of the possibility of building a server with commodity hardware at home on which you could run AI applications that could run in production and be competitive with the big hyperscalers. The possibility of developing sophisticated AI applications, system and infrastructure accessible and affordable to everyone. The possibility that everyone can participate in the current innovation cycle of AI.

And all that is possible with open source hardware and software. You can get second hand/used hardware/server components from marketplaces like ebay. The last or 2nd last generation of hardware is still good for most ML/DL/AI applications. And there are many startups out there that are trying to make AI more accessible to the masses as well.

Being smart about building these systems can help a lot of people participating in the space. You just need to know about the opportunities that are out there. Whether it be the cheap secondary market server hardware on eBay, or the open source software that is used in AI operations or benchmarks of how different system configurations perform with different models, algorithms and ML/AI/DL-frameworks like pytorch or tensorflow and how different AI algo’s perform in production for different use cases on different system configurations.

The problem I see is that there is a lot of information out there about AI systems and infrastructure, but there is not one place you can go to and learn how these systems perform and which components are needed to build AI systems, which new innovation came out that will make these system more affordable and efficient and how all these things perform in production (based on real life benchmarks). That information is nowhere to be found and that’s the information I want to provide in my blog.

---

What you will find in this blog:

• benchmarks of AI frameworks/models on different system configurations
• benchmarks of new hardware (CPUs, GPUs, NPUs, Memory, etc.)
• new hardware innovation, it’s location in the AI system architecture, and which efficiency gain it brings
• general overview of the architecture of AI systems and Infrastructure
  • how are AI systems built, what is needed to build the most basic AI system?
  • what software is needed to build AI applications
• What’s available now, today! And what will come in future.