insight
Kubrick Business Insights & Analytics Engineer Timothy Driscoll explores the tools and platforms on display at this year’s AWS Summit, where generative AI capability took center stage. He examines the rise of AI-powered tools for transforming data engineering and analytics, with a range of products from AWS, and the need for nuance to drive outputs that are fine-tuned to user requirements.
If there was just one takeaway from this year's AWS Summit NYC, it would be that generative AI is the future. It has the potential to completely revolutionize how all types of organizations get things done, and it already has to some degree. Right now, generative AI is helping professionals to write emails and summarize documents, allowing students to create summaries and get new explanations for complex topics, and even calling into question the future outputs of Hollywood writers[1]. However (for the time being at least), it appears that generative AI is simply helping employees become more productive and would be unable to completely replace people. While LLMs are powerful tools which are trained on vast amounts of data, they can still demonstrate a lack of true, human understanding necessary for creative and nuanced tasks.
While use cases for business users across industries are still being developed, the most immediate impact of generative AI is closer to home: within the data and technology functions that understand its capability best. It is in this domain that AWS have unveiled a series of products to make their mark in this new market. In particular, AWS have made a huge bet on organizations and individuals wanting to fine-tune their own specific AI models, rather than trying to make a single perfect, all-purpose model. Because of this, AWS has focused on building easy-to-use interfaces and infrastructure around AI rather than training one expansive model, which would be costly and more prone to producing inconsistencies and lack of nuance. In doing so, AWS allows – and is actively encouraging – businesses to build, train, and deploy their foundational models using their platform, with capability to test strengths and weaknesses unique to their own business challenges and the data at their disposal.
In addition to the build-it-yourself capability, the AWS Summit showcased some of the plug-and-play generative AI tools that AWS has created for the needs of the programmer. While ChatGPT has dominated the market for text generation, and disrupters like stability.ai have burst onto the scene for image generation, AWS’ Codewhisperer is their ‘AI coding companion’, which has helped coders configure solutions 57% faster on average. Moreover, Codewhisperer is fine-tuned to work extremely well with AWS Glue, a data integration tool that enables data pipelining from across locations to enable and accelerate modelling and analytics. Instead of spending countless hours reading documentation and learning syntax of new tech, data engineers, data analysts, and data scientists can use Codewhisperer to spend more time on configuring the best solution to their problem.
One AWS platform on prominent display at the Summit was Amazon Bedrock, a service which makes foundational models available through an API. At the live demo of their beta version, they showcased how users can configure connections to their organization’s different proprietary datasets to utilize an LLM which can retrieve data points based on written prompts. Thus, they empower analysts to request and receive data specific to their challenge in their own language, no querying required.
Meanwhile, Amazon Opensearch allows for similar capabilities, but with vector databases. For organizations which store their data in vector embeddings, Opensearch can be used to train LLM chatbots to easily access their data by running searches with the vector engine. Going one step further in automating the role of the analyst, Amazon QuickSight demonstrates the power of AI to create data analytics dashboards. Built using the capability of Amazon Bedrock and powered by natural language prompts, you can now generate a business intelligence (BI) dashboard that tells the story of your data – a form of ‘Generative BI’, as Amazon have coined it.
AI is undoubtedly changing the way we work, and AWS are forging the path to make the adoption of AI for a variety of use cases a reality. Whether it is accessing data through Amazon Bedrock and Amazon Opensearch, generating dashboards with Amazon QuickSight, or having your own personal assistant to undertake and validate your work with Amazon Codewhisperer, every data and technology professional should be aware of the plethora of tools at their disposal. But the race to unlock the potential of AI is not a lone venture for Amazon; every major technology vendor and disruptor is seeking new ways to help businesses harness the power of AI. So, amidst the noise and excitement, the onus is on data and technology professionals to understand their needs to find the right tools for them. Furthermore, and perhaps more importantly, we must understand the limitations of these tools and simultaneously recognize and champion the skills and knowledge that human users bring to work in unison with Gen AI in order to create the greatest value for all.
At Kubrick, we help organizations to evolve with next-generation technology with the skills that deliver change and impact. Our unique approach to hiring and training talent from a wide range of backgrounds allows us to build a workforce equipped with today’s most sought-after skills at the intersection of data, AI, and cloud. As a Partner of AWS, Kubrick are supporting organizations across industries to embrace cloud technology to drive scale, efficiencies, and new analytics and AI capability.
To learn more about how Kubrick can support your team to navigate the changing technology landscape, get in touch: speaktous@kubrickgroup.com
