OpenSearchCon 2024: North America – The blog post

Mon, Dec 09, 2024 · Parth Shah, Mikhail Smelik

OpenSearch, the community-driven, open-source search and analytics suite, has been gaining momentum in the data-driven application landscape. The recent OpenSearchCon 2024: North America event held in San Francisco, which attracted over 350 attendees from around the world, was a testament to the growing influence of this powerful platform. In this blog post, we’ll explore the key highlights, insightful sessions, and exciting announcements that emerged from this year’s conference.

Keynote: The future of OpenSearch

The event kicked off with an inspiring keynote from Eli Fisher, Sr. Manager, Product Management at OpenSearch, who laid out the vision for the future of OpenSearch. Some key announcements included:

  • OpenSearch 2.x and unlocking AI potential: Eli shared details about the OpenSearch 2.x release, including features like the OpenSearch AI toolkit, a built-in retrieval-augmented generation (RAG) processor, and concurrent segment search, unlocking new possibilities for AI-powered applications.
  • OpenSearch generative AI use cases: Anupam Chaturvedi, Adobe Engineering Manager, demonstrated how Adobe is leveraging OpenSearch for its Acrobat generative AI assistant, showcasing the practical applications of OpenSearch in the generative AI space.
  • Community growth: Jim Zemlin, Executive Director, The Linux Foundation, spoke about welcoming OpenSearch to the Linux Foundation. By opening up project governance and moving to a neutral home, the OpenSearch Project is taking concrete steps to further the project’s founding principle of being for the community and by the community.

Important sessions and workshops

The conference featured several informative sessions and workshops, including:

  • Session: Airbnb Embedding Platform This session, led by Moutupsi Paul, Xiaotang Wang, and Amulya Sharma, focused on how Airbnb is leveraging OpenSearch as a vector database for its embedding platform. The session provided an overview of how Airbnb is using vector embeddings to solve guest, host, and marketplace challenges, followed by specific use cases that led Airbnb to adopt OpenSearch as its vector database. The session shared lessons learned and provided insights into strategies for optimizing OpenSearch as a vector database.
  • Session: Lucene And Beyond - Core Storage Extension In OpenSearch OpenSearch is tightly bound to the Lucene core APIs that facilitate encoding, transactions, merges, and search. This session, led by Samuel Herman, focused on how OpenSearch storage encoding can be extended to popular formats (for example, Parquet, Avro) that are readable by public big data systems such as Apache Spark.
  • Session: Bringing cloud native innovation from search platforms at Uber and Slack into OpenSearch This session, led by Karthikeyan Ramasamy and Bryan Burkholder, showcased how their organizations’ unique requirements of low-latency, high-throughput workloads led them to develop their own search and log analytics platforms, making it easy to operate, cost effective, and able to scale to petabytes of data. The session covered how Uber and Slack are collaborating with the OpenSearch Project to bring some of their learnings and innovations from their platforms into OpenSearch, with a special focus on a modular cloud-native architecture featuring isolation between readers and writers and a performant messaging protocol for communication between services.
  • Session: A deep dive into OpenSearch Serverless This session, led by Rohit Nair and Milav Shah, dives deep into how Amazon OpenSearch Serverless works, including architecture details, understanding performance and operational characteristics, and the future roadmap.

Closing remarks: What’s next for OpenSearch?

The event was not just about technical sessions—it was also a fantastic opportunity to network with peers in the open-source search space. The community lounge was buzzing with activity, and there were plenty of informal meetups where attendees could connect with OpenSearch engineers, contributors, and advocates.

As the OpenSearchCon event drew to a close, it was clear that the conference was a significant milestone for the OpenSearch community. The announcements about new features, improvements in observability, and the ongoing commitment to open-source principles left attendees excited about the future of the project.

The OpenSearch roadmap for 2024 highlights key themes around which contributors are innovating, including stability, availability, and resiliency (StAR); cost, performance, and scale (CoPS); ease of use; observability and log analytics; search and machine learning (ML); security; and a modular architecture.

As the OpenSearch Project continues to grow, the community is streamlining the process of creating a roadmap that includes ideas and projects provided directly by the community. It’s clear that the project is poised to make an even bigger impact in the world of AI/ML, search, and observability. We will be back in 2025 with three different OpenSearchCon events, including in Europe, India, and North America. You can bookmark the Upcoming Events page and consider attending one of the project’s bi-weekly meetups or an in-person user group. We encourage you to try out OpenSearch, engage with the community, and stay tuned for the exciting developments to come.