Generative AI: A Magic Tool for Businesses?

Lately, it's nearly impossible to go a day without encountering headlines about generative AI technologies and their applications like ChatGPT or AutoGPT. AI has become red hot again, and its hotness is astonishing: suddenly almost everyone wants to jump on the AI bandwagon.

As an advocate for the power of large language models (LLMs), I believe that generative AI carries immense potential. In fact, these models have already demonstrated their practical value in enhancing personal productivity. For instance, I have incorporated code generated by the models in my work and even used GPT-4 to proofread this blog post.

The pressing question now is: how can businesses, small or large, which aren't involved in the creation of LLMs, capitalize on the power of generative AI to improve their bottom line?

Unfortunately, there is a chasm between using LLMs for personal productivity gain and monetizing LLMs for business profit. Like developing any business solution, there is much more than meets the eye. Just using the example of creating a chatbot solution with GPT-4, it can easily take months and cost millions of dollars, not to mention the required deep AI expertise and engineering skills.

So organizations must decide whether they want to heavily invest in in-house development or collaborate with a vendor. With many businesses including many enterprises having scarce AI and IT resources, in-house development may not be an option. Their only option would be working with a vendor. But how to evaluate and choose the right one?

The rest of my article will outline the challenges and opportunities to leverage generative AI for business gains, unveiling the lay of the AI land for entrepreneurs, corporate executives, and investors alike looking to unlock the value of generative AI in business context.

Since my focus is on generative AI for business gains, I will start with the business expectations of AI and the valuable lessons learned from AI blunders to better manage such expectations.

Business Expectations of AI

In the modern business landscape, technology is an integral part of business. When a business adopts a technology, it expects the technology to improve its operation efficiency and drive better business outcomes. Businesses expect of AI to do the same, no matter which type of AI is.

Moreover, the success of business does not solely depend on technology. A well-run business will continue to prosper, and a poorly managed one will still struggle, regardless of the emergence of generative AI or tools like ChatGPT.

In general, a successful adoption of AI for business requires that AI MUST perform to deliver concrete business value as expected and the business must know how to manage AI just like managing any other business operations.

Disillusionments of Generative AI

While generative AI has certainly advanced the field of AI especially in natural language understanding (NLU) and natural language generation (NLG), it cannot solve every business problem. Nor will it do. To avoid investing in generative AI hypes, I'd like to first pinpoint out two common disillusionments about generative AI and recognize the associated practical challenges of adopting generative AI for business.

Disillusionment #1

A common disillusionment is that generative AI technologies like GPT or generative AI tools like ChatGPT level the playing field for every business: anyone can use generative AI and English becomes the new programming language.

While this may be true for certain content creation use cases (e.g., marketing copywriting and information summary), generative AI is incapable of solving complex business problems that require deep domain knowledge. For example, ChatGPT generates a medical article with "significant inaccuracies", and it also failed a CFA exam.

While domain experts have the in-depth domain knowledge, they may not be AI savvy or understand the inners of how generative AI works. For example, they may not know how to prompt ChatGPT effectively or refine their prompts to obtain the desired results.

So the challenge is how to enable business domain experts to train and supervise AI but without requiring them to become IT/AI experts.

Disillusionment #2

Another common disillusionment related to generative AI is that any IT personnel can use generative AI technologies like GPT to build awesome business software solutions. While generative AI may help accelerate software development (e.g., writing sample code), all the nitty gritty software development requirements, such as user experience design, data security, system scaling and performance, still need to be met and generative AI's help is rather limited.

This reminds me of the emergence of internal combustion engine, which made car manufacturing possible. While all mechanical engineers or craftsman had access to the engine, not everyone became, nor did every car maker have the success as Ford Motors did.

This also reminds me of the emergence of microprocessor (CPU), which made the making of personal computers (PCs) possible. While virtually all computer engineers and craftsman had access to microprocessors, not everyone became Steve Jobs, nor did every PC maker achieve the success of Apple.

History teaches us while new technology inventions can generate great business opportunities, the opportunities always favor those with deep expertise and passion in the related field: Ford's expertise and passion for mechanics enabled him to capitalize the opportunity brought by internal combustion engine and Job's expertise and obsession with design and user experience enabled him to capitalize the opportunity brought by microprocessor.

As a new technology, generative AI is no exception. Those with deep AI and IT expertise will benefit from generative AI the most and the fastest by capitalizing on the opportunities.

So the challenge is if your organization does not have the expertise to capitalize on the opportunities brought by generative AI, whom should you partner to harness its power?

Valuable Lessons from AI Blunders

The concept of AI has been around since 1960's and the adoption of AI has been through valleys and peaks. One thing agreed among AI experts is that AI is FAR from perfect, including generative AI.

For example, a well-known and challenging issue with generative AI is "hallucination", with AI generating false or harmful output.

Despite rapid improvements in the power and quality of generative AI, advancements in truthfulness have been incremental. While the debate over whether this is a fundamental weakness of generative AI continues among academics, a look at the current state reveals that even the latest state-of-the-art models, like Dromedary, which outperforms GPT-4, can only achieve less than 70% accuracy on the TruthfulQA benchmark. This level of accuracy is not sufficient for most business settings. In other words, no businesses would want to foot the bill for the liability of giving a false response to their customers or prospects 30% of time!

Additionally, there are concerns surrounding privacy and security. For instance, Samsung recently banned the use of ChatGPT due to data leak and Italy also banned the use of ChatGPT due to data privacy.

The experiences and lessons learned from past AI deployments are still applicable and valuable, as they faced similar challenges. Discarding the past efforts to encode domain knowledge and the tools developed to handle complex business cases would be wasteful.

The current wave of generative AI can be likened to a more powerful CPU. If you have already built a computer, you can simply plug in the new CPU to boost performance. This is faster than building a computer from scratch around a new processor. When selecting a vendor, it's helpful to see if they have a track record of success. The decision essentially boils down to whether you trust someone who has never built a computer to utilize the new powerful CPU, or if you prefer someone with prior experience in building computers.

Technical Ingredients for Successful AI Adoption

As the CTO of an AI company, I have witnessed firsthand the challenges and rewards of deploying generative AI in real-world business settings. Based on my experience, I have identified three key areas of technical competence that are crucial for a successful implementation of generative AI solutions.

1. Agent Framework

An agent does not only respond to user requests, it may also take actions proactively on users behalf. An effective agent framework forms the foundation of a generative AI system, allowing it to communicate and interact with users and other systems in a meaningful way. This framework should encompass several components:

• Perceptors and actuators: These essential components serve as the AI system's eyes, ears, mouth and hands, enabling it to perceive the environment and take actions accordingly. For instance, perceptors might include natural language processing capabilities, while actuators could involve generating text or controlling other software tools. Generate AI excels in this aspect.
• Control Unit: This component should be able to coordinate the perceptors and actuators. A well-designed control unit is vital for managing natural language understanding, generation, and reasoning within the AI system. The power, expressiveness and flexibility of the control unit is critical. There are some general open source implmentation of control unit, such as LangChain, but many use cases require customized control unit.
• Working memory: Just like humans, AI systems need a working memory to maintain context during interactions. This component should be able to store and retrieve short-term information, allowing the AI system to respond accurately and consistently to user inputs. In order to process and manage the vast amounts of high-dimensional data generated by AI systems, a vector database is essential for building working memory. This database should be optimized for handling embeddings and other complex data structures, ensuring that the AI system can efficiently access and utilize the information it needs.
• Customizable Plugins: Seamless integration with third-party tools and services is crucial for maximizing the AI system's capabilities. By incorporating external resources, the system can expand its functionality and offer a more comprehensive solution to users. These extensions should enable access to customer data and provide additional functionality, ensuring that the AI system remains adaptable and relevant in a variety of use-cases.

2. No-Code Platform

In addition to the large amount of training data, the success of ChatGPT is largely due to the " Reinforcement Learning from Human Feedback" (RLHF), highlighting the importance of incorporating human input in AI solutions. To ensure a seamless collaboration between AI and human expertise, a no-code platform should be implemented.

• Human UI: A user-friendly, no-code interface allows business users to interact with and manage the AI system without requiring programming expertise. This empowers non-technical personnel to harness the power of AI, enabling a broader adoption of the technology within the organization.
• Test and Evaluation : Comprehensive testing and evaluation processes should be in place to ensure that the AI system meets quality standards and business requirements. Regular evaluations help to identify areas for improvement and drive the ongoing development of the AI system. Tools such as dashboard visualization should be provided to facilitate these processes.
• Live human integration: In situations where the AI system is unable to provide a satisfactory response or solution, a smooth integration with live human agents is essential. This ensures that businesses can consistently deliver the desired outcomes and maintain a high level of customer satisfaction.

3. Infrastructure

A robust infrastructure is the backbone of a successful AI deployment, supporting the system's performance, security, and scalability:

• DevOps and Scaling : A DevOps approach to AI deployment facilitates streamlined system maintenance, updates, and scaling. By closely integrating development and operations teams, businesses can rapidly adapt to changing demands and ensure that their AI system remains agile and responsive.
• Security and Compliance : Ensuring the security and compliance of the AI system is crucial for maintaining trust and confidence among users. Businesses must prioritize safeguarding sensitive data and adhering to industry-specific regulations, demonstrating their commitment to protecting user privacy and upholding ethical AI practices.

Conclusion

As generative AI continues to revolutionize the business landscape, choosing the right AI vendor becomes increasingly crucial for organizations looking to harness the technology's full potential. To make an informed decision, businesses should evaluate potential vendors based on their competence in three critical areas: an effective agent framework, a seamless no-code platform for human-AI collaboration, and a robust infrastructure to support performance, security, and scalability.

By carefully selecting a vendor with proven expertise in these domains, organizations can ensure a successful deployment of generative AI solutions tailored to their specific needs. This, in turn, will help enterprises drive innovation, increase productivity, and gain a competitive edge in an increasingly AI-driven market.

In conclusion, as businesses navigate the rapidly evolving world of generative AI, partnering with a vendor that excels in these three key areas will pave the way for success. By diligently evaluating potential partners and their capabilities, enterprises can make strategic decisions that unlock the power of generative AI and turn it into gold for their organization.

Note

Extensions of this post appear in two articles on VentureBeat and CXSCOOP