Frequently Asked Questions

1.1 What is OpenSearch?

OpenSearch is a fully open source search and analytics suite. OpenSearch includes OpenSearch (derived from Elasticsearch 7.10.2) and OpenSearch Dashboards (derived from Kibana 7.10.2), and a range of features like built-in security, anomaly detection, alerting, observability experiences, and more.

1.2 Why was OpenSearch created?

Developers embrace open source software for many reasons, one of the most important is the freedom to use that software where and how they wish. Elastic ceased making open source options available for Elasticsearch and Kibana, releasing them under the Elastic license, with source code available under the Elastic License or SSPL. These are not open source and do not offer users the same freedoms. Because of this, we made the decision to create a fork from the last Apache 2.0 version of Elasticsearch and Kibana and provide OpenSearch under the Apache License, Version 2.0 (ALv2).

ALv2 grants well-understood and permissive usage rights that match the freedoms people expect with open source software; freedoms such as being able to use, modify, extend, monetize, and resell the open source software where and how they want. For OpenSearch, we believe this will enable broad adoption and contributions benefiting all members of the community.

1.3 Why should I use OpenSearch?

OpenSearch enables people to easily ingest, secure, search, aggregate, view, and analyze data. These capabilities have led it to be popular for use cases such as application search, log analytics, and more. With OpenSearch you benefit from having an open source product you can use, modify, extend, monetize, and resell how you want. At the same time, we will continue to provide a secure, high-quality search and analytics suite with a rich roadmap of new and innovative functionality.

1.4 Is OpenSearch suitable for production use?

OpenSearch became ready for production use in July of 2021 with the generally available release of OpenSearch 1.0.

1.5 What license is OpenSearch released under?

All of the software in the OpenSearch project is released under the Apache License, Version 2.0 (ALv2). The ALv2 license grants you well-understood usage rights for OpenSearch. You can use, modify, extend, embed, monetize, resell, and offer OpenSearch as part of your products and services. We have also published permissive usage guidelines for the OpenSearch trademark, so you can use the name to promote your offerings.

1.6 Is AWS equipped to maintain and advance a project like OpenSearch?

When AWS decides to offer a service based on an open source project, we ensure that we are equipped and prepared to maintain it ourselves, if necessary. We bring years of experience working with Elasticsearch and Kibana codebases and have made upstream code contributions to both Elasticsearch and Apache Lucene (the core search library that Elasticsearch is built on). We have added several features in open source like security, alerting, anomaly detection, index state management, and trace analytics that are widely used and deployed in production by our community and customers. We are well equipped to maintain and advance the project ourselves. Also, the community-backed codebase will help accelerate new innovations and will allow everyone to move faster in improving stability, scalability, resiliency, and performance. Already many organizations including SAP, CapitalOne, RedHat, Logz.io, Aiven.io, Bonsai, Logit.io, Search Guard, and BAInsight have publicly backed OpenSearch.

1.7 What is the relationship between the existing Open Distro for Elasticsearch project and OpenSearch?

The final version of Open Distro for Elasticsearch is version 1.13. Future versions of the plugins and advanced features distributed with Open Distro for Elasticsearch will be available in the OpenSearch project.

1.8 Is OpenSearch wire-compatible with Elasticsearch?

Yes. OpenSearch is a fork of open source Elasticsearch 7.10. As such, it provides backwards REST APIs for ingest, search, and management. The query syntax and responses are also the same. In addition, OpenSearch can use indices from Elasticsearch versions 6.0 up to 7.10. We also aim to support the existing Elasticsearch clients that work with Elasticsearch 7.10.

Note that while the OpenSearch API is backwards compatible, some clients or tools may include code, such as version checks, that may cause the client or tool to not work with OpenSearch.

For more information on backwards compatibility, see upgrading FAQs.

1.9 Will the OpenSearch API change in future versions?

All future OpenSearch 1.x releases will be backwards compatible with Elasticsearch 7.10, although some APIs will be deprecated. If functionality requires a breaking change, we will introduce a new major version of OpenSearch and provide tooling to make migrating to the new major version simple. When new features require adding APIs, we will work with the community to add support for these features in popular clients.

For more details on deprecated APIs, see upgrading FAQs.

1.10 Do Elasticsearch clients like Logstash and Beats work with OpenSearch?

Since OpenSearch is wire-compatible with Elasticsearch 7.10, any clients that currently work with Elasticsearch 7.10 should also work with OpenSearch. For details specific about open source Logstash and Beats compatibility please see the documentation.

Please report any issues you experience with these clients or other clients in our project GitHub issues.

For more questions related to upgrading clients, see upgrading FAQs.

1.11 Does OpenSearch include forks of Logstash and Beats?

No. There is documentation on which versions of Logstash and Beats work with OpenSearch. We will be providing, more documentation on how to use them with OpenSearch and download links to OpenSearch compatible versions.

1.12 Can I upgrade from Elasticsearch to OpenSearch?

Yes. You can upgrade using a rolling upgrade (one node at a time) process when upgrading from Elasticsearch versions 7.0 - 7.10 to OpenSearch. For Elasticsearch versions 6.x you will be required to perform a cluster restart upgrade. OpenSearch can use indices from Elasticsearch versions 6.0 up to 7.10. Indices on versions prior to Elasticsearch 6.0 or after 7.10 will need to be removed from a cluster being upgraded to OpenSearch or reindexed into a compatible version of Elasticsearch then upgraded to OpenSearch.

1.13 Which features of Elasticsearch do you plan to offer in the future? Is OpenSearch going to keep pace with the upstream Elasticsearch releases? How will this evolve?

The future investments and plans for OpenSearch should be viewed as independent of Elasticsearch. Our roadmap will be driven by the needs of the community. Where possible, if new features in OpenSearch are similar to Elasticsearch, we will strive to make the APIs similar. The two projects, however, are distinct. The OpenSearch project is open source and is focused on providing the innovations that our community and customers ask for.

1.14 Can OpenSearch read indices from previous versions of Elasticsearch?

Yes. OpenSearch is compatible with indices created from Elasticsearch versions 6.0 up to 7.10.

1.15 Will you be making contributions back to Elasticsearch open-source?

No. Open source Elasticsearch development ended with 7.10 when it was moved to a non-open source license. OpenSearch software will move forward according to the needs of the community. That said, all OpenSearch software built for its customers and the community is released under the ALv2 license and we welcome anyone to use the software under the ALv2 terms and conditions.

1.16 I want to contribute, what can I do?

We welcome contributions in multiple forms. You can submit pull requests, open issues, and leave comments on any of the OpenSearch GitHub repositories. You can join our community meetings. You can also leave comments in the community forum. If you have a specific way you would like to contribute and don’t know what steps to take, please leave a comment in our community forum.

1.17 How are new features, bug fixes, and other development decisions made?

New software developed for OpenSearch is made based on project’s principles for development. To learn more, visit our website.

1.18 What will your governance model be? Are you looking at any foundations?

At this time, there is not a plan to move OpenSearch in to a foundation. As we work together in the open, we expect to uncover the best ways to collaborate and empower all interested stakeholders to share in decision making. Cultivating the right governance approach for an open source project requires thoughtful deliberation with the community. We’re confident that we can find the best approach together over time. For now, AWS is the steward of OpenSearch. The principles of development define the guidelines for how decisions about the project are made. These principles will continue to be iterated and refined based on feedback.

1.19 How do I contribute functionality and get it on the public roadmap?

If you want to add something to OpenSearch that is not in the public roadmap, that’s a perfect opportunity to contribute! There are a couple of things you can do.

You can create a feature request in the relevant GitHub repo for the feature and socialize the request. People are always looking for in-demand features to build. A maintainer or someone else in the community may pick this feature up and work on it. As progress is made the maintainers of the repo will help get the feature onto the roadmap.

Another option is to build the feature yourself. To do this create a proposal as a GitHub issue in the relevant repo and use the proposal template (thanks jkowall for contributing the template!). Offer your commitment to build it. The maintainers of the repo will work with you to figure out how best to proceed. That could be further discussion, design docs, or just starting to write the code. As the feature is developed, the maintainers of the repo will also work with you to incorporate it into the roadmap.

1.20 What tools do you recommend for log and metrics collection?

OpenSearch is supported by a range of tools like Beats, Fluentd, Fluent Bit, and OpenTelemetry Collector. Moving forward, we will focus effort on improving Data Prepper, Fluentd, and FluentBit. Users who are using Beats <= 7.12.x as an agent tool, and considering open source alternatives, should migrate to Fluent Bit >= 1.9 or Open Telemetry Collector. Beats version >= 7.13 does not support OpenSearch.

1.21 What tools do you recommend for log aggregation?

OpenSearch is supported by a range of tools like Data Prepper, Fluentd, Logstash, and Kafka. OpenSearch believes in multiple open source options and will focus on improving the Data Prepper, Fluentd, and Kafka support going forward.

2.1 Can I use OpenSearch plugins with the proprietary Elastic stack?

The plugins are tested to work with OpenSearch. They have not been tested with Elastic’s proprietary software. As these plugins are open source, we do welcome anyone who wants to test them out with the Elastic Stack. However, we do not plan to invest in making the OpenSearch plugins work on the Elastic stack.

2.2 Can I install an OpenSearch plugin as a standalone plugin?

Yes. You can install an OpenSearch plugin independently of the other plugins. For example, if you would like to use OpenSearch with only our security plugin installed, you can remove the other plugins using the OpenSearch plugin remove command.

2.3 Will older versions of Open Distro for Elasticsearch plugins still be available for open source Elasticsearch?

No. Open Distro has been archived.

2.4 Does OpenSearch include forks of Logstash and Beats?

No. For more information on which versions of Logstash and Beats work with OpenSearch, see the Compatibility Matrix for Logstash.

2.5 What is OpenSearch Data Prepper?

Data Prepper is a server-side data collector capable of filtering, enriching, transforming, normalizing and aggregating data for downstream analytics and visualization. Also, Data Prepper lets users build custom pipelines to improve the operational view of applications.

Two common uses of Data Prepper are trace and log analytics. Trace analytics help you visualize the flow of events and identify performance problems. Log analytics can improve search functionality, as well as help you analyze and provide insights into your application.

To get started building your own custom pipelines with Data Prepper, see the Data Prepper Get Started guide.

2.6 What is Fluentd and Fluent Bit?

Fluentd is a data collector for log data collection, processing, and forwarding. It’s written in Ruby and supports over 500 plugins including data sources, data output, parsers, formatters, and filters.

Fluent Bit is an agent that collects data from different sources, enriches them with filters, and sends them to multiple destinations. It’s designed with performance in mind, meaning it is optimized for high throughput and low CPU and memory usage. It’s written in C and has an architecture that supports more than 70 plugins for inputs, filters, and outputs.

3.1 How can I upgrade an Elasticsearch OSS and Kibana OSS cluster with multiple nodes to OpenSearch and OpenSearch Dashboards?

OpenSearch supports rolling upgrades in the same way as Elasticsearch OSS. You can deploy OpenSearch into a mixed cluster with Elasticsearch OSS or Open Distro for Elasticsearch nodes. One by one you can replace the legacy nodes with little to no additional manual work.

In the same way as Kibana OSS, OpenSearch Dashboards does not support rolling upgrades, but it supports restart upgrades. You are able to stop all Kibana OSS instances, deploy a new OpenSearch Dashboards instance and direct traffic to it.

3.2 How can I upgrade a single node deployment of Elasticsearch OSS and Kibana OSS to OpenSearch and OpenSearch Dashboards?

You are able to stop Elasticsearch OSS and Kibana OSS, install OpenSearch and OpenSearch Dashboards, manually configure those to point to your Elasticsearch OSS and Kibana OSS data, review and potentially update settings, then start OpenSearch with OpenSearch Dashboards.

3.3 Which versions of Elasticsearch OSS and Kibana OSS can I upgrade from to OpenSearch and OpenSearch Dashboards, directly?

You can directly upgrade to OpenSearch 1.x from Elasticsearch OSS and Kibana OSS 6.8.0-7.10.2, and Open Distro 1.x. For more information about upgrades, please see the documentation.

3.4 Can I upgrade older versions of Elasticsearch OSS and Kibana OSS to OpenSearch and OpenSearch Dashboards?

Elasticsearch OSS and Kibana OSS 5.x up to 6.7.2 can be first upgraded to 6.8.0, then it is recommended to upgrade to Elasticsearch OSS 7.10.2 or ODFE 1.13, before upgrading to OpenSearch and OpenSearch Dashboards.

Note that the minimum supported index version for OpenSearch is 6.0. So, all the 5.x indices have to be re-indexed, before upgrading to OpenSearch.

3.5 Can I upgrade a non-OSS Elasticsearch and Kibana cluster to OpenSearch?

Yes. However, functionality not available in Elasticsearch OSS and Kibana OSS continues to not be available. We recommend evaluating additional features available in OpenSearch that may provide similar functionality.

3.6 Can I upgrade from Elasticsearch and Kibana 7.11 or other versions released after the fork?

No, this will not be supported in OpenSearch 1.0.

3.7 Will my ODFE plugins continue to work after upgrade?

OpenSearch plugins based on the Open Distro for Elasticsearch (ODFE) plugins are included in OpenSearch 1.0, and are functionally backwards compatible with their predecessors.

3.8 At what point is the upgrade process complete?

When all nodes are running OpenSearch and OpenSearch Dashboards 1.0.

3.9 Does an upgrade to OpenSearch require downtime?

You can perform rolling upgrades from Elasticsearch OSS to OpenSearch, which does not require downtime. Kibana OSS upgrades require a restart which will cause downtime for Kibana OSS and OpenSearch Dashboards. If you have a single-node deployment and wish to upgrade it in-place, you will incur downtime.

3.10 Can I do a Blue/Green upgrade with data migration?

Yes. You can choose to do a Blue/Green upgrade instead of a rolling upgrade.

3.11 Can I install elasticsearch-xyz-plugin in OpenSearch?

An Elasticsearch plugin that depends on Elasticsearch JARs will not work without code changes to depend on OpenSearch JARs.

3.12 Can I rollback an upgrade half-way in progress (e.g. 2/4 nodes are running OpenSearch)?

This is possible and has caveats. For example, upgrading Kibana OSS to OpenSearch Dashboards migrates the Kibana OSS indexes and does not allow rollback.

3.13 Are there any changes required for my existing Elasticsearch OSS clients to continue to work?

While the OpenSearch API is backwards compatible, some clients or tools may include code, such as version checks, that may cause the client or tool to not work with OpenSearch.

3.14 Can I run a heterogeneous mixed OpenSearch and Elasticsearch OSS cluster?

This only supported in OpenSearch for the purpose of a rolling upgrade.

3.15 Can I run a heterogeneous mixed OpenSearch Dashboards and Kibana cluster?

No. This is not supported.

3.16 Is OpenSearch security enabled by default after an upgrade?

Yes. Security is enabled by default.

3.17 What settings change during the upgrade?

Environment variables that contain branded words such as ES or OPENDISTRO have been renamed.

Cluster settings that contain branded words such as es. or opendistro. can continue to be used but have been deprecated.

3.18 What REST APIs change during the upgrade?

REST APIs that contain branded words such as ES or OPENDISTRO have been deprecated.

3.19 How do I upgrade or migrate secure or system indices?

There is no need to migrate secure or system indices for the upgrade to OpenSearch 1.0. This is because theses indices are not being renamed. In future upgrades, index aliases or migrations may be introduced for secure or system indices.

3.20 Is there binary compatibility between Elasticsearch-OSS and OpenSearch?

While an OpenSearch node is able to join an Elasticsearch OSS cluster, namespaces and class names in OpenSearch have been changed. If your plugin code depends on Elasticsearch OSS JARs, you will need to upgrade those dependencies to OpenSearch JARs.

3.21 Where can I find more information and track progress on ensuring backwards compatibility?

The umbrella issue for backwards-compatibility is OpenSearch#671. Issues across opensearch-project repositories are labeled backwards-compatibility.

3.22 Will Kibana work on top of OpenSearch 1.0?

No. You will need to upgrade to OpenSearch Dashboards.

3.23 I have built a custom plugin for Elasticsearch, will it run without modification on OpenSearch?

Because OpenSearch is backwards compatible, it will. If you find identify an incompatibility, please open an issue in OpenSearch.

3.24 Can I install kibana-xyz-plugin that only uses REST APIs in OpenSearch Dashboards 1.0?

While the APIs have not changed, we have not tested kibana-xyz-plugin.

3.25 I have built a custom visualization for Kibana, will it be usable with OpenSearch Dashboards without modification?

Because OpenSearch Dashboards is backwards compatible, it will. If you find identify an incompatibility, please open an issue in OpenSearch Dashboards.

3.26 Can OpenSearch read indices from previous versions of Elasticsearch?

Yes. OpenSearch is compatible with indices created from Elasticsearch versions 6.0 up to 7.10.

3.27 Will the OpenSearch API change in future versions?

All future OpenSearch 1.x releases will be backwards compatible with open source Elasticsearch 7.10. If functionality requires a breaking change, we will introduce a new major version of OpenSearch and provide tooling to make migrating to the new major version simple. When new features require adding APIs, we will work with the community to add support for these features in popular clients.

3.28 Which version of OpenSearch should I use?

For OpenSearch and other software in the OpenSearch project, new features and active development always takes place against the newest version. The OpenSearch project follows the semantic versioning specification for assigning version numbers to releases, so in most cases you should be able to upgrade to the latest version of any software without encountering incompatible changes.

Sometimes an incompatible change is unavoidable. When this happens, the software’s maintainers will increment the major version number (e.g. increment from OpenSearch 1.y.z to OpenSearch 2.0.0), and the most recent release of the previous major version will be put into maintenance. During the maintenance window, the software will continue to receive bug fixes and security patches, but no new features.

The duration of the maintenance window will vary from product to product and release to release. It will be based on the patching and support schedules for dependencies the software includes, community input, the scope of the changes introduced by the new version, and estimates for the effort required to continue maintenance of the previous version.

3.29 Can I use tools that require a particular version of Elasticsearch?

By default, OpenSearch reports its version number, however OpenSearch can be configured to report 7.10.2 for compatibility with existing tools designed for Elasticsearch versioning. This configuration option is deprecated and will log a deprecation message and will be removed in the future in a major version of OpenSearch (3.0.0 or later).

3.30 Does OpenSearch support version downgrades?

OpenSearch does not support direct version downgrades. If your environment must be downgraded, we recommend using snapshots to create a restore point, then restoring the snapshot to a cluster built with the target version.

4.1 How do I add a project to the community project page on opensearch.org?

To add a project:

• Fork the opensearch website repository.
• Create a branch. The recommended naming convention is adds-<your project name>-to-community-projects
• Add a markdown file for your project to ./_community_projects/your-project.md
• Last create a pull request and one of the website maintainers will review it and work with you to merge it.

To see an example pull request, look here).

4.2 What kinds of projects are welcome on the community project page?

Any project that benefits the OpenSearch community is welcome. These can be both open source licensed projects and proprietary licensed projects.

4.3 What are the requirements for a project before it can be moved into the OpenSearch Project GitHub organization?

There are three requirements:

• The project is licensed under a permissive open source license, like Apache 2.0.
• The project has broad utility for the OpenSearch community
• The project is currently actively maintained

4.4 What are the benefits of moving a project under the OpenSearch Project GitHub organization?

The main benefits include increased visibility, and if you no longer can or want to maintain the project some day, the project will help find new maintainers to transition the ownership. Everything else remains the same, including that you retain the administrator permissions of your GitHub repository

4.5 How can I get my project into the OpenSearch Project GitHub organization?

If you are interested in moving your project, create an issue in your repository, visibly confirm with your co-maintainers that you would like to make the move, and tag dblock and elfisher to initiate the process of moving your project.

4.6 What is an OpenSearch Project Github organization repository administrator?

An admin owns stewardship of the repository and its settings. Admins have admin-level permissions on a repository. Use those privileges to serve the community and protect the repository as follows. Administrator responsibilities can be found in the OpenSearch .github repository admins file.

4.7 What is an OpenSearch Project Github organization repository maintainer?

Maintainers are active and visible members of the community, and have maintain-level permissions on a repository. They use those privileges to serve the community and evolve the software in the repository they maintain. Maintainer responsibilities can be found in the OpenSearch .github repository maintainer file.

4.8 What are my security obligations when developing in OpenSearch Project Github organization repositories?

Security is your number one priority. Maintainer’s Github keys must be password protected securely and any reported security vulnerabilities are addressed before features or bugs. Note that repositories in the OpenSearch GitHub organization are monitored and supported 24/7 by Amazon Security, see Reporting a Vulnerability for details.

4.9 How should an OpenSearch Project Github organization repository roadmap be managed?

Maintainers can manage the repository roadmap in the way that best works for them. An example roadmap/project board can be found here.

4.10 How do I contribute to an OpenSearch Project Github organization repository?

It’s strongly recommend that you follow the contributors guide.