Difference between revisions of "Stackalytics/HowToRun"
(→Prerequistics) |
Julien-zte (talk | contribs) m (→Installation) |
||
(30 intermediate revisions by 7 users not shown) | |||
Line 3: | Line 3: | ||
Stackalytics depends on the following applications: | Stackalytics depends on the following applications: | ||
* Python-pip Python Package Index | * Python-pip Python Package Index | ||
+ | * libpython2.7-dev - development C headers for Python | ||
+ | * libssl-dev - SSL development libraries, header files and documentation | ||
* Git - used to retrieve project repos and get their log | * Git - used to retrieve project repos and get their log | ||
* Memcached - used to store commits | * Memcached - used to store commits | ||
− | |||
* Nginx - scalable web server | * Nginx - scalable web server | ||
− | * Uwsgi - application container for Nginx | + | * Uwsgi, uwsgi-plugin-python - application container for Nginx and plugin for Python |
Web dashboard is made on Flask framework. Typical deployment of Flask application uses nginx as front-end server and uwsgi as app container. For development purposes Stackalytics may be run on DevStack. The project doesn't require any libraries not listed in OpenStack requirements project. | Web dashboard is made on Flask framework. Typical deployment of Flask application uses nginx as front-end server and uwsgi as app container. For development purposes Stackalytics may be run on DevStack. The project doesn't require any libraries not listed in OpenStack requirements project. | ||
Line 16: | Line 17: | ||
1. Install necessary applications | 1. Install necessary applications | ||
− | |||
− | 2. Create folders needed for Stackalytics | + | Particular command depends on linux distribution. Here is example for Debian based system: |
+ | sudo apt-get install libpython2.7-dev libssl-dev python-pip git memcached nginx uwsgi uwsgi-plugin-python | ||
+ | |||
+ | 2. Configure memcached | ||
+ | |||
+ | Add two options to startup script: | ||
+ | -M Disable automatic removal of items from the cache when out of memory. Additions will not be possible until adequate space is freed up. | ||
+ | -m 1024 Use <num> MB memory max to use for object storage; the default is 64 megabytes. | ||
+ | |||
+ | ''Note: On Debian/Ubuntu systems using the memcached package, this configuration is stored in /etc/memcached.conf'' | ||
+ | |||
+ | 3. Create a stackalytics user and group | ||
+ | |||
+ | sudo useradd -U stackalytics | ||
+ | |||
+ | 4. Create folders needed for Stackalytics | ||
mkdir /opt/stack/ | mkdir /opt/stack/ | ||
− | chown | + | chown stackalytics.stackalytics /opt/stack |
mkdir /var/local/stackalytics | mkdir /var/local/stackalytics | ||
− | chown | + | chown stackalytics.stackalytics /var/local/stackalytics |
− | The last folder is a place where all repos are checked out. | + | The last folder is a place where all repos are checked out. 'stackalytics' is an account that will be used for Git and Gerrit polling. |
− | + | 5. Check out Stackalytics code | |
cd /opt/stack/ | cd /opt/stack/ | ||
− | git clone | + | git clone https://github.com/openstack/stackalytics.git |
− | + | 6. Install Stackalytics: | |
cd /opt/stack/stackalytics | cd /opt/stack/stackalytics | ||
− | sudo | + | sudo pip install . |
This will install Stackalytics and all needed libraries into the system. | This will install Stackalytics and all needed libraries into the system. | ||
− | + | ''Note: On Debian/Ubuntu systems the dependencies for package cffi require installing the distro package libffi-dev, and package cryptography requires installing distro package libssl-dev. Without these, the pip install command will fail." | |
− | |||
− | + | 7. Configure OpenStack gerrit access | |
− | + | StackAlytics requires an account to access the Gerrit review system. Refer to the documentation at [http://docs.openstack.org/infra/manual/developers.html the Developer's Guide] for details on setting up an account. | |
+ | |||
+ | The private key and SSH username used to authenticate with Gerrit are configured in the file /opt/stack/stackalytics/etc/stackalytics.conf with the following parameters: | ||
+ | |||
+ | # SSH key for gerrit review system access | ||
+ | # ssh_key_filename = /home/user/.ssh/id_rsa | ||
+ | |||
+ | # SSH username for gerrit review system access | ||
+ | # ssh_username = user | ||
+ | |||
+ | 8. Process all registered repos: | ||
stackalytics-processor | stackalytics-processor | ||
This command checks out all repos and parses them. Debugging may be enabled by passing --debug parameter (--verbose for error debugging) | This command checks out all repos and parses them. Debugging may be enabled by passing --debug parameter (--verbose for error debugging) | ||
− | + | If customized configuration options are used, be sure to specify the configuration file on the command line, as in: | |
+ | stackalytics-processor --config-file /opt/stack/stackalytics/etc/stackalytics.conf | ||
+ | |||
+ | 9. Run dashboard in development mode: | ||
stackalytics-dashboard | stackalytics-dashboard | ||
− | This command starts dashboard to listen on port | + | This command starts dashboard to listen on port 8080 of the localhost. The port may be changed by --listen-port parameter. |
== Production deployment == | == Production deployment == | ||
Line 66: | Line 93: | ||
</uwsgi> | </uwsgi> | ||
</pre> | </pre> | ||
+ | |||
+ | ''Note'': On Debian/Ubuntu systems using the native uwsgi package installation, this configuration should be written to /etc/uwsgi/apps-enabled/stackalytics.xml. Then uwsgi needs to be restarted with 'sudo service uwsgi restart' | ||
+ | |||
nginx config: | nginx config: | ||
Line 76: | Line 106: | ||
} | } | ||
location /static/ { | location /static/ { | ||
− | alias /opt/stack/stackalytics/dashboard/static/; | + | alias /opt/stack/stackalytics/stackalytics/dashboard/static/; |
} | } | ||
} | } | ||
</pre> | </pre> | ||
+ | |||
+ | ''Note'': On Debian/Ubuntu systems using the native nginx package installation, this configuration should be written to /etc/nginx/sites-available/default, or a separate config file in /etc/nginx/sites-available/ Then nginx needs to be restarted with 'sudo service nginx restart' |
Latest revision as of 15:12, 5 May 2017
Prerequistics
Stackalytics depends on the following applications:
* Python-pip Python Package Index * libpython2.7-dev - development C headers for Python * libssl-dev - SSL development libraries, header files and documentation * Git - used to retrieve project repos and get their log * Memcached - used to store commits * Nginx - scalable web server * Uwsgi, uwsgi-plugin-python - application container for Nginx and plugin for Python
Web dashboard is made on Flask framework. Typical deployment of Flask application uses nginx as front-end server and uwsgi as app container. For development purposes Stackalytics may be run on DevStack. The project doesn't require any libraries not listed in OpenStack requirements project.
Installation
Stackalytics deployment is verified on both Ubuntu and CentOS, however the first option is preferable since it has all necessary components as system packages.
1. Install necessary applications
Particular command depends on linux distribution. Here is example for Debian based system:
sudo apt-get install libpython2.7-dev libssl-dev python-pip git memcached nginx uwsgi uwsgi-plugin-python
2. Configure memcached
Add two options to startup script:
-M Disable automatic removal of items from the cache when out of memory. Additions will not be possible until adequate space is freed up. -m 1024 Use <num> MB memory max to use for object storage; the default is 64 megabytes.
Note: On Debian/Ubuntu systems using the memcached package, this configuration is stored in /etc/memcached.conf
3. Create a stackalytics user and group
sudo useradd -U stackalytics
4. Create folders needed for Stackalytics
mkdir /opt/stack/ chown stackalytics.stackalytics /opt/stack mkdir /var/local/stackalytics chown stackalytics.stackalytics /var/local/stackalytics
The last folder is a place where all repos are checked out. 'stackalytics' is an account that will be used for Git and Gerrit polling.
5. Check out Stackalytics code
cd /opt/stack/ git clone https://github.com/openstack/stackalytics.git
6. Install Stackalytics:
cd /opt/stack/stackalytics sudo pip install .
This will install Stackalytics and all needed libraries into the system.
Note: On Debian/Ubuntu systems the dependencies for package cffi require installing the distro package libffi-dev, and package cryptography requires installing distro package libssl-dev. Without these, the pip install command will fail."
7. Configure OpenStack gerrit access
StackAlytics requires an account to access the Gerrit review system. Refer to the documentation at the Developer's Guide for details on setting up an account.
The private key and SSH username used to authenticate with Gerrit are configured in the file /opt/stack/stackalytics/etc/stackalytics.conf with the following parameters:
# SSH key for gerrit review system access # ssh_key_filename = /home/user/.ssh/id_rsa # SSH username for gerrit review system access # ssh_username = user
8. Process all registered repos:
stackalytics-processor
This command checks out all repos and parses them. Debugging may be enabled by passing --debug parameter (--verbose for error debugging)
If customized configuration options are used, be sure to specify the configuration file on the command line, as in:
stackalytics-processor --config-file /opt/stack/stackalytics/etc/stackalytics.conf
9. Run dashboard in development mode:
stackalytics-dashboard
This command starts dashboard to listen on port 8080 of the localhost. The port may be changed by --listen-port parameter.
Production deployment
Production deployment is done with help of uwsgi and nginx.
uwsgi config:
<uwsgi> <socket>/tmp/stackalytics.sock</socket> <pythonpath>/opt/stack/stackalytics/dashboard/</pythonpath> <module>web:app</module> <plugins>python27</plugins> <env>STACKALYTICS_CONF=/usr/local/etc/stackalytics/stackalytics.conf</env> </uwsgi>
Note: On Debian/Ubuntu systems using the native uwsgi package installation, this configuration should be written to /etc/uwsgi/apps-enabled/stackalytics.xml. Then uwsgi needs to be restarted with 'sudo service uwsgi restart'
nginx config:
server { listen 80; location / { uwsgi_pass unix:///tmp/stackalytics.sock; include uwsgi_params; } location /static/ { alias /opt/stack/stackalytics/stackalytics/dashboard/static/; } }
Note: On Debian/Ubuntu systems using the native nginx package installation, this configuration should be written to /etc/nginx/sites-available/default, or a separate config file in /etc/nginx/sites-available/ Then nginx needs to be restarted with 'sudo service nginx restart'