Jump to: navigation, search

Difference between revisions of "Searchlight"

(Concept Internals Overview)
m (Contact)
(26 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 +
[[File:OpenStack Project Searchlight vertical.png|frameless]]
  
 
= Mission Statement =
 
= Mission Statement =
Line 7: Line 8:
  
 
{| border="1" cellpadding="2"
 
{| border="1" cellpadding="2"
| Source code (current - forked from Glance)
+
| Developer Documentation
| https://github.com/lakshmisampath/searchlight/
+
| https://docs.openstack.org/searchlight/latest/
 
|-
 
|-
| Source code (proposed)
+
| API Reference
 +
| http://developer.openstack.org/api-ref/search/
 +
|-
 +
| Source code - API and Listener Services
 
| https://github.com/openstack/searchlight
 
| https://github.com/openstack/searchlight
 +
|-
 +
| Source code - Horizon UI Plugin
 +
| https://github.com/openstack/searchlight-ui
 +
|-
 +
| Source code - Python Client
 +
| https://github.com/openstack/python-searchlightclient
 +
|-
 +
| Gerrit Reviews
 +
| https://review.openstack.org/#/q/project:%255E.*searchlight.*+status:open,n,z
 
|-
 
|-
 
| Bug tracker
 
| Bug tracker
| https://bugs.launchpad.net/searchlight
+
| https://storyboard.openstack.org/#!/project_group/searchlight
 
|-
 
|-
| Feature tracker (see Glance for historical tracking)
+
| Feature tracker
| https://blueprints.launchpad.net/searchlight
+
| https://storyboard.openstack.org/#!/project_group/searchlight
 
|-
 
|-
 
| IRC
 
| IRC
Line 23: Line 36:
 
|-
 
|-
 
| Meeting Times
 
| Meeting Times
| http://eavesdrop.openstack.org/#Search_Team_Meeting
+
| http://eavesdrop.openstack.org/#Searchlight_Team_Meeting
 
|-
 
|-
 
| Meeting Agenda
 
| Meeting Agenda
Line 34: Line 47:
 
= Overview =
 
= Overview =
  
Searchlight was originally developed and released in the Kilo release of Glance as the Catalog Index Service [1]. At the Liberty Summit we decided to broaden the scope to provide advanced and scalable search across multi-tenant cloud resources.
+
Searchlight dramatically improves the user-focused search capabilities and performance on behalf of various OpenStack cloud services.
  
[1] http://specs.openstack.org/openstack/glance-specs/specs/kilo/catalog-index-service.html
+
It accomplishes this by offloading user search queries from existing API servers and indexing their data into ElasticSearch. ElasticSearch is a search server based on Lucene. It provides a distributed, scalable, near real-time, faceted, multitenant-capable full-text search engine with a RESTful web interface and schema-free JSON documents. ElasticSearch is developed and released as open source under the terms of the Apache License. Notable users of ElasticSearch include Wikimedia, StumbleUpon, Mozilla, Quora, Foursquare, Etsy, SoundCloud, GitHub, FDA, CERN, and Stack Exchange. (Source: http://en.wikipedia.org/wiki/Elasticsearch). The elastic-recheck project also uses Elasticsearch (and kibana) to classify and track OpenStack gate failures. (Source: http://status.openstack.org/elastic-recheck)
 +
 
 +
The current focus of Searchlight is to extend the search to a multi-cloud / Edge environment. The tenant can search for authorized resources and services on other providers'  cloud or container platforms (see User cases). VIM and Orchestrator can use Searchlight APIs to query resource status and service requirement for their management operations.
  
This is intended to dramatically improving the search capabilities and performance of various OpenStack cloud services.
+
=== Use cases ===
  
It accomplishes this by offloading user search queries from existing API servers by indexing their data into ElasticSearch. ElasticSearch is a search server based on Lucene. It provides a distributed, scalable, near real-time, faceted, multitenant-capable full-text search engine with a RESTful web interface and schema-free JSON documents. ElasticSearch is developed and released as open source under the terms of the Apache License. Notable users of ElasticSearch include Wikimedia, StumbleUpon, Mozilla, Quora, Foursquare, Etsy, SoundCloud, GitHub, FDA, CERN, and Stack Exchange. (Source: http://en.wikipedia.org/wiki/Elasticsearch). The elastic-recheck project also uses Elasticsearch (and kibana) to classify and track OpenStack gate failures. (Source: http://status.openstack.org/elastic-recheck)
+
Check out this documentation: https://docs.openstack.org/searchlight/latest/user/usecases.html
  
 
=== Screencasts ===
 
=== Screencasts ===
  
To help explain the ideas of the project, we have a quick screencast demonstrating the concepts. This was done by taking the Glance Catalog Index Service and adding in a plugin for Nova and then modifying horizon to use it. This is what was demonstrated at the Liberty Design Summit in both a Glance fishbowl session and a Horizon fishbowl session.
+
* (Mitaka) Integration with Horizon demoː https://www.youtube.com/watch?v=2feC1njvZe0
 
+
* (Mitaka) End of Cycle Presentation on Horizon, CLI, and Searchlightː https://www.youtube.com/watch?v=ExzULavwvNQ
* https://youtu.be/eGnGr48E5_4
+
* (Mitaka) Summit Presentationː https://www.youtube.com/watch?v=0jYXsK4j26s
 +
* (Liberty) PTL Overviewː https://www.youtube.com/watch?v=yU5CrAOAlkA
 +
* (Kilo summit) Concept Demoː https://youtu.be/eGnGr48E5_4
  
 
=== Design ===
 
=== Design ===
  
The design is based off the Catalog Index Service in Glance. It will be refined moving forward as cross project needs are discovered and defined.
+
* https://docs.openstack.org/searchlight/latest/
 
+
* https://docs.openstack.org/searchlight/latest/admin/architecture.html
* Glance Specificationː http://specs.openstack.org/openstack/glance-specs/specs/kilo/catalog-index-service.html
 
  
 
==== Concept Overview ====
 
==== Concept Overview ====
Line 62: Line 78:
 
* [https://wiki.openstack.org/w/images/5/55/Searchlight-Concept-2.png Concept Internals]
 
* [https://wiki.openstack.org/w/images/5/55/Searchlight-Concept-2.png Concept Internals]
 
* [https://wiki.openstack.org/w/images/e/ef/Searchlight-use-when-there.png Usage switching]
 
* [https://wiki.openstack.org/w/images/e/ef/Searchlight-use-when-there.png Usage switching]
 +
* [https://wiki.openstack.org/w/images/1/10/Searchlight-Concept-Horizon-Layers.png Horizon Concept Layers]
 +
* [https://wiki.openstack.org/w/images/6/6b/Searchlight-WebSocket-Concept.png Horizon Web Socket Concept]
 +
 +
==== Multi-cloud  ====
 +
* [[Searchlight multi-cloud analysis]]
  
 
== Get Involved ==
 
== Get Involved ==
  
Please join with us to help move forward together as a community! We are sure that the ideas and concepts can use refinement and we'd like to identify where we can best fit in to the ecosystem.
+
Please join with us to help move forward together as a communityǃ
 +
 
 +
Searchlight is an open project and we encourage contribution from everybody.
 +
 
 +
We support both developers and non-developers who want to provide input, requests for features, and bug fixes. We want to be able to move quickly without getting too bogged down in the process but still provide a rich mechanism for feature reviews as needed.
 +
 
 +
* https://docs.openstack.org/searchlight/latest/contributor/feature-requests-bugs.html
 +
 
 +
== Contact ==
 +
 
 +
PTL: Trinh Nguyen <dangtrinhnt [at] gmail [dot] com>
 +
 
 +
Contributors:
 +
 
 +
* Sa Pham <tbd[at] gmail [dot] com>
 +
* Thuy Dang <thuydang.de[at] gmail [dot] com>
 +
 
 +
 
 +
Past Contributors:
 +
 
 +
* TBD
 +
 
 +
== History ==
 +
 
 +
Searchlight was originally developed and released in the Kilo release of Glance as the Catalog Index Service [1]. At the Liberty Summit, we decided to broaden the scope to provide advanced and scalable search across multi-tenant cloud resources.
 +
 
 +
[1] http://specs.openstack.org/openstack/glance-specs/specs/kilo/catalog-index-service.html

Revision as of 09:02, 25 January 2019

OpenStack Project Searchlight vertical.png

Mission Statement

To provide advanced and scalable indexing and search across multi-tenant cloud resources.

Project Links

Developer Documentation https://docs.openstack.org/searchlight/latest/
API Reference http://developer.openstack.org/api-ref/search/
Source code - API and Listener Services https://github.com/openstack/searchlight
Source code - Horizon UI Plugin https://github.com/openstack/searchlight-ui
Source code - Python Client https://github.com/openstack/python-searchlightclient
Gerrit Reviews https://review.openstack.org/#/q/project:%255E.*searchlight.*+status:open,n,z
Bug tracker https://storyboard.openstack.org/#!/project_group/searchlight
Feature tracker https://storyboard.openstack.org/#!/project_group/searchlight
IRC #openstack-searchlight
Meeting Times http://eavesdrop.openstack.org/#Searchlight_Team_Meeting
Meeting Agenda https://etherpad.openstack.org/p/search-team-meeting-agenda
Meeting Logs http://eavesdrop.openstack.org/meetings/openstack_search/

Overview

Searchlight dramatically improves the user-focused search capabilities and performance on behalf of various OpenStack cloud services.

It accomplishes this by offloading user search queries from existing API servers and indexing their data into ElasticSearch. ElasticSearch is a search server based on Lucene. It provides a distributed, scalable, near real-time, faceted, multitenant-capable full-text search engine with a RESTful web interface and schema-free JSON documents. ElasticSearch is developed and released as open source under the terms of the Apache License. Notable users of ElasticSearch include Wikimedia, StumbleUpon, Mozilla, Quora, Foursquare, Etsy, SoundCloud, GitHub, FDA, CERN, and Stack Exchange. (Source: http://en.wikipedia.org/wiki/Elasticsearch). The elastic-recheck project also uses Elasticsearch (and kibana) to classify and track OpenStack gate failures. (Source: http://status.openstack.org/elastic-recheck)

The current focus of Searchlight is to extend the search to a multi-cloud / Edge environment. The tenant can search for authorized resources and services on other providers' cloud or container platforms (see User cases). VIM and Orchestrator can use Searchlight APIs to query resource status and service requirement for their management operations.

Use cases

Check out this documentation: https://docs.openstack.org/searchlight/latest/user/usecases.html

Screencasts

Design

Concept Overview

Searchlight-Concept-1.png

Concept Internals Overview

Multi-cloud

Get Involved

Please join with us to help move forward together as a communityǃ

Searchlight is an open project and we encourage contribution from everybody.

We support both developers and non-developers who want to provide input, requests for features, and bug fixes. We want to be able to move quickly without getting too bogged down in the process but still provide a rich mechanism for feature reviews as needed.

Contact

PTL: Trinh Nguyen <dangtrinhnt [at] gmail [dot] com>

Contributors:

  • Sa Pham <tbd[at] gmail [dot] com>
  • Thuy Dang <thuydang.de[at] gmail [dot] com>


Past Contributors:

  • TBD

History

Searchlight was originally developed and released in the Kilo release of Glance as the Catalog Index Service [1]. At the Liberty Summit, we decided to broaden the scope to provide advanced and scalable search across multi-tenant cloud resources.

[1] http://specs.openstack.org/openstack/glance-specs/specs/kilo/catalog-index-service.html