Our managing partner, Alireza Ali, sat down again with The Startup Podcast to discuss the latest news and trends in AI. The conversation touched topics such as Llama3, Claude Opus, YC W24 batch and more. Check it out here:

Whether monolithic AGI or systems-style AI built from components will ultimately dominate is still very much an  open question and something we’re paying close attention to. Berkeley AI Research recently wrote about the concept they term “Compound AI”. The premise is surprisingly simple, yet (in our view) quite unsurprising! The main idea is that in order to get the most out of the latest language, code and image models, you need to lego-brick various components together to build a performant system that really does what you need it to do. We’re really not at the point yet of having a single model or even interface that can solve complex business or enterprise (or consumer) problems. “Agent RAG” is another term we’ve seen that captures a subset of these ideas. Put another way, few if any of the existing off-the-shelf models or tools will be sufficient to really solve the problem you want to address with an AI system. By bringing together a variety of purpose-built tools and models, you’ll be forming a system that does what you really want it to do. There are an incredible variety and amount of pre-trained models, tools and frameworks available today, as well as hosted models & services from cloud providers. Many are trained for specific tasks such as general language modeling, instruction following or text-to-image production. Even in the language space, there are a variety of tools and components that make up some of the most performant systems, the “RAG” paradigm being the most well known and combining a neural retrieval system with a generative LLM. But, RAG is only one such compound system, and there are a variety of configuration and optimization variants just in RAG solutions alone. Other possible compound systems include dynamically generated prompts, iterative prompt optimization, fact-checking filters and more. Optimization is always problem-specific… Optimization is almost always an ongoing process, and is very specific to the problem and goals of the system. Yes, many pretrained models are optimized for particular tasks and some are even optimized for, well, better general performance…but, the highest value applications are usually very specific and can only be measured end to end. If you’re building a financial analyst agent, you can really only evaluate the quality based on the end goals you care about: is it analyst accuracy? Investment ROI? Errors in predictions? You may think that having to determine which components to use and chain together sounds like a lot of work, and that can be true. Conversely, the compositional nature of today’s AI tools is really an incredible development that offers a high degree of control over cost, performance and the specifics of the compound solution. Luckily, there are also a number of tools and frameworks looking to join the sub-components and make it easier to fit them together. The BAIR blog post names a number of these, including Databricks AI Gateway for a ‘model router’, LlamaIndex for RAG and other tool-use compositing as well as DSPy for prompt and model optimization based on a ‘golden set’ or evaluation set you have (although DSPy works without one). What does ‘Compound AI’ mean for you and your business? Compound AI re-emphasizes the big unresolved question of how the AI shift will play out: in favor of Big-Tech consolidators? Or, with truly democratized, a-la-carte Open Source solutions? Compound AI essentially says, “we still don’t know” and also, that no one has built the one-model-to-rule-them all. Until that’s been achieved, the incredible compositionality of commercial and open source/weight models and tools mean that this is a great time to be building AI solutions, for yourself or for others. If you’d like to talk about how Compound AI ideas may play into your AI strategy or planning, please reach out to us at info@theblue.earth - we love to brainstorm AI solutions against the backdrop of all the tools and models available now.

What’s behind the huge amount of attention going to LLMs like ChatGPT, Llama and Gemini? Rich text representations that go way beyond “straight” encodings like ASCII and Unicode. Rich text representations capture the reality of knowing a word by the company it keeps. These representations are kind of the compressed context of a word or phrase. They are the key to LLM's abilities to produce meaningful language in response to prompts. Indeed, rich text encodings like word embeddings pull the representations directly from activations within a neural network trained on a recovery or prediction task. But, word embeddings (rich text representations) can have a wide variety of applications that may be much better suited to your task than wide-open LLM interfaces like ChatGPT. Semantic Clustering Why do we care about rich text representations and what do we mean by ‘semantics’ ? Equivalent meaning between words and phrases is something humans are good at, but computers have not been great at until recently. Human judgment about whether phrases or documents “mean” the “same”, are similar or dis-similar accounts for a huge amount of human effort in the workplace. Semantic clustering and related tools can significantly amplify the productivity of the humans doing work involving organization, categorization and consolidation. Highly paid financial analysts spend much of their time scouring earnings reports and press releases, ultimately in order to make a judgment or reconcile contradictory information. Analysis and judgment benefits from the experience and high-order pattern matching of the analyst. The reading and initial grouping and organization of areas of focus, however, (for example, new product prospects, supplier bottleneck risks, competitive landscape) could be greatly sped up and made much more accurate through semantic clustering tools. source: https://txt.cohere.com/combing-for-insight-in-10-000-hacker-news-posts-with-text-clustering How much do end-users need to know about the workings of these tools and methods? We would say, a rough understanding of how these encodings are derived (by learning context relationships from a dataset and a prediction or recovery deep-learning task) can be extremely helpful. Of course, you can make use of these tools without knowing anything about how they are derived, but even a conceptual understanding may allow for more creative use and application of the tools as well as their limitations. “Today’s semantic understanding tools allow for post-analysis of existing data without making significant or even any changes to business processes.” Business Intelligence of a few years ago understood the value of detailed analytics on customer feedback and interactions, but often required complex changes to business processes to ‘code’ and track interactions in a way that could be counted and trended over time. Very few organizations were successful in transforming their business processes to successfully enable this. Data drift and data pollution are extremely common, further lowering the value of these types of enterprise data. Today’s semantic understanding tools allow for (and may even get more value from) post-analysis of existing data without making significant or even any changes to business processes. Textual or recorded (audio) natural language data contains the real judgment and input of humans all along the value chain, from the ideation, production and eventual consumption of consumer goods & services. In a way, this data is much more “pure” than data which Business Intelligence deployers created through coded menus and drop-downs trying to capture anticipated categories of interactions. Not only does this constrain and distort what gets reported, the work of accurate coding on top of existing processes is often dehumanizing and boring! This exacerbates the problem of drift and pollution through “encouraged” laziness and shortcuts. (Imagine the Verizon customer support rep who has to help you fix your bill, and simultaneously choose the cause of your trouble from some long-winded drop-down menu so BI can drive marketing or product changes months from now…) Semantic clustering can be used for a wide variety of text (or speech-to-text) data cleanup, categorization, classification and similarity tasks. Indeed, many language AI projects are best begun by first analyzing clusters of the text you ultimately want to work with (whether that’s building search, conversational or business intelligence systems). Narrow tasks like survey response analysis, customer feedback analysis, social media trending and customer support triage can all greatly benefit from an initial (and iterative) round of semantic clustering. And, there are great tools that can take advantage of the latest embedding models and clustering approaches. If you’re wondering how semantic clustering can help your business or would like to signup for a free 45 mins consultation session, reach out to us at info@theblue.earth

Emil is out this week from the Insider trio so Yaniv and Chris welcome special guest Alireza Ali to discuss the hottest developments in the tech industry this week. Ali is currently the co-founder and partner at Blue Dot who specializes in AI strategy and implementation consulting services. He previous founded another AI startup and has worked at Google for 11 years, some of which as a Tech Lead. Together, this week’s trio discusses: 🌌OpenAI's Sora 🧠Groq's AI chip 💥Gemini's competitive 1.5 release 📈AI Startup Trends Stay up with the latest developments in the AI scene. Link: https://www.tsp.show/reacts-ai-alignment-openai-sora-gemini-15-ai-startup-trends/

Who needs an AI strategy? What does it look like? Do I really need one? It sounds like I need to hire lots of PhDs and enter an entirely unfamiliar and very fast-moving space…Yikes! Our position is twofold: Everyone (almost everyone) needs an AI strategy today. An AI strategy can be simple and easy An AI strategy can be very simple. It can also be complex, but our main contention is that it really doesn’t have to be. The main thing is to start now, if you haven’t already. And, if you have started, keep reading to either affirm or adjust your strategy - we have some suggestions for how to simplify thinking about AI strategy. We believe that AI is the next phase of the information age. As personal and desktop computers came, followed by the internet,  AI is the next phase, and its nature is quite different from previous phases.  It is here, but it is also still forming and evolving, and there are huge opportunities for the right type of adoption and strategy. So the ‘why’ of AI strategy is that AI is coming very quickly, and that a certain amount of AI expertise will very quickly (if not already) become table stakes for almost all businesses and industries. The other ‘why’ of AI strategy is that it may end up being more personalized and fragmented than previous rounds of information technologies. Of course, big players will have offerings, and may end up having the best and cheapest ones. But, it would be unwise to just wait and see…Taking ownership and control of your AI strategy, even if it is a simple one, or one based on existing platforms, will put you in a much better position. What is AI strategy? Is it complex and expensive? Does it mean lots of PhDs and data analysis and virtual-reality headsets? A simple suggestion to start…AI strategy means understanding conceptual AI capabilities and learning to match them to business opportunities (and challenges). Your initial AI strategy may be as simple as just trying out some “canned” AI tools like ChatGPT or Gemini if you haven’t already. Taking things a bit further, you could include trying some tools on your company data to get a sense of what’s possible and what’s coming. If you don’t think you have relevant company data, or don’t think you can get it easily, then try with some of your personal data. We’ll make a recommendation below. Seeing this in action even on your own data can provide a lot of hints as to what’s possible. The main idea is to start to build confidence and familiarity with the abstractions around AI and LLMs beyond simply reading about them in popular business and tech press. Take Search & Recommendation as an example You may think Search is a solved problem - we’re already on to the next thing. Google solved Search for the internet, and AI is beyond Search. But, the reality is that lots of today’s AI (by which we mean deep learning AI like LLMs and image captioning) is still ultimately about better (smarter) search. The thing that’s changed is that search systems have gotten much better - specifically, they understand what you’re looking for much better than before, and they understand the content you’re looking for much better than before. Most AI systems today are about much better matching of your search intent and content, whether that’s a webpage, a photo, a full-length movie or your email. And, another theme that will just continue to repeat itself is AI learning much more complex relationships than the previous generation’s search systems…Better email or photo search is enabled by learning much richer relationships between the words or words and images, and making these learned systems easy to query. Indeed, early Google search really was a form of AI and language modeling. So, our first suggestion is to think of AI as advanced search that understands human language better. We sometimes (always?) forget how Google trained us to search using keywords like “best gifts golfer men 2023”. What is this language? We certainly don’t talk to people this way.  Getting computers to understand questions (and their answers) that are much closer to the nuance and manner in which we communicate with each other is a major key to understanding today’s AI capabilities (including those that can be applied to your business, products, data or personal life). “think of AI as advanced search” Imagine all the things you could do with advanced search and you’ve made a huge step forward in having a conceptual understanding of deep learning AI and its applications. “Find me all the highest value sales leads that resulted in closed sales over the past 5 years” or “Give me a list of all the products we made in the past 10 years that had the lowest return rate and highest review ratings”. This might just sound like “Business Intelligence” that was all the rage 5-10 years ago. Except that today’s AI systems (search systems) can now handle many of these types of analyses without investing in huge systems, hiring teams of consultants to scrub and host databases or specialists to build and manage these systems. What kind of search system would you love to have for your business or customers? If you don’t have business data to test with or it’s not easy to get, Dropbox makes a product that allows you to use AI-based search across all your personal content: email, bookmarks, documents, etc. Recommendation is another capability that is suddenly very powerful and accessible and can match many business tasks that you might not have considered. In a way, recommendation is just another type of search, but a specialized one. Yes, sophisticated product and content recommendation systems are now much easier to build from free or packaged AI software offerings, and you might have considered these beyond your reach or too complex to deploy. But, the abstraction of recommendation systems may suggest other applications. Where might you have opportunities to find similarities between customer types? You may currently class customer types by their industry (education, government, manufacturing). But, recommendation and similarity systems might find a better classification of clients that can be used for smarter or higher-conversion upsell. An example might be that your clients are better grouped by the features they prioritize or value. Recommendation can be used to find these hidden relationships, but also to recommend upsell offerings to the “right” customers. Deep-learning-based semantic clustering and classification on data that’s never been analyzed before might contain gold in terms of customer insights and opportunities. AI strategy is largely about understanding the building blocks of AI systems at the right level of abstraction, and then looking for places where you can apply these capabilities. AI strategy also involves making hard decisions about investment timelines and costs/benefit analysis. Given the extremely fast pace of change in not only the underlying technologies, but also the startup and software landscape of providers, this can be an overwhelming task, but we believe it’s an important area to spend time on. We’ll have more to say about this in the future, but you’ll be reassured to know that very few people are confident in this area right now. The most important thing you can do is start to build some experience and confidence at the lowest cost and risk that you can. As you start to develop familiarity with the tools, capabilities and abstractions, you’ll be able to make better and bigger investment decisions more confidently. There are so many tools and options with lots of great documentation. That’s a blessing and a curse as the pure volume and hype can make it hard to make a decision. Getting help from other companies and products should definitely be in consideration, but ultimately, how you decide to employ AI in your business strategy should be your decision - taking ownership of this decision confidently is one of the most important things you can do right now.