Mercurial > sci
view man/sci.7 @ 71:ef8f9c986080
scid: fix worker with . in their names
author | David Demelier <markand@malikania.fr> |
---|---|
date | Tue, 24 Jan 2023 10:04:47 +0100 |
parents | 5076be758687 |
children | 71cd8447e3a4 |
line wrap: on
line source
.\" .\" Copyright (c) 2021-2022 David Demelier <markand@malikania.fr> .\" .\" Permission to use, copy, modify, and/or distribute this software for any .\" purpose with or without fee is hereby granted, provided that the above .\" copyright notice and this permission notice appear in all copies. .\" .\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES .\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF .\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR .\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES .\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN .\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF .\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. .\" .Dd August 04, 2022 .Dt SCI 7 .Os .\" NAME .Sh NAME .Nm sci .Nd simple continuous integration framework .\" DESCRIPTION .Sh DESCRIPTION The .Nm framework is a set of utilities to run automated tasks similarly to alternatives such as jenkins or buildbot. .Pp In contrast to those, .Nm sci does not know how to build project nor how to retrieve information when to build them. It only works by user project scripts to be executed upon addition of jobs. .Pp It is designed in mind to be as simple as possible to improve flexibility and simple documentation. .\" OVERVIEW .Sh OVERVIEW The .Nm framework is split into three individual programs that are used independently. The communication workflow is: .Bd -literal o ---- scictl / scid (HTTP) o | \\ | o ---- sciworkerd SQLite .Ed .Pp The .Nm scid daemon is the unique access to the SQLite database and simply take requests over an HTTP REST API to retrieve and set results into it. .Pp The .Nm scictl is the administrative utility to update the database using a command line interface. It can also be used to create jobs and their result manually if wanted. .Pp The .Nm sciworkerd is a daemon executing jobs on a host machine. It access the jobs listing by querying .Nm scid and output their result. .\" ENTITIES .Sh ENTITIES The process handles different kind of entities in the database. .\" PROJECTS .Ss PROJECTS A project is a user description of what to be automated and tested. It has a name, description, project URL and a script to execute. They can be created using the .Cm project-add command from .Nm scictl . .\" JOBS .Ss JOBS Jobs are tasks to be performed by any worker for a given project. It has an user arbitrary tag that will be passed to the project script as sole argument. In contrats to many CI system, the sci framework has no information about how to build and access a project and as such the job tag can be anything up to the user (a SCM repo revision, date, simple id, etc). .\" WORKERS .Ss WORKERS A worker is a host system that connects to .Nm scid using HTTP protocol to get acces to jobs to perform, execute them and finally send the result back. They have been designed to use HTTP to allow remote usage. .\" JOB RESULTS .Ss JOB RESULTS A job result is the detail about a job ran by a worker for a specific project. If a job exists for one project and there are four workers on the user installation, there will be four job results. It has an exit code (got from the user script), a log console (capture from standard output and error) and a timestamp when it was started. .\" GETTING STARTED .Sh GETTING STARTED To setup the .Nm framework you need at least: .Bl -enum .It A daemon .Xr scid 8 running and accessible remotely. .It One or more .Xr sciworkerd 8 running on a machine that can access .Xr scid 8 . .It Add some projects and register those workers using .Xr scictl 8 utility. .El .\" Setup scid .Ss Setup scid The .Xr scid 8 daemon does not require much configuration, you can specify the database file to use with its appropriate options. Otherwise, you can simply run the daemon with no arguments. It will initialize the database and generate a default API key that .Xr scictl 8 and .Xr sciworkerd 8 require to perform requests. Use the .Cm api-get command to get that key. .Pp The .Xr scid 8 program isn't a daemon by itself but a CGI or FastCGI process which needs to be coupled with a dedicated web server. For the example let's use nginx and the .Xr kfcgi 8 utility to spawn our FastCGI process. .Pp Configure the nginx server to include at least the following code snippet for a specific virtual host (the .Xr scid 8 process explicitly requires its own virtual host). .Bd -literal -offset indent server { server_name sci.myhostname.fr; listen 80; location / { fastcgi_param QUERY_STRING query_string; fastcgi_param REQUEST_METHOD $request_method; fastcgi_param CONTENT_TYPE $content_type; fastcgi_param CONTENT_LENGTH $content_length; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; fastcgi_param SCRIPT_NAME $fastcgi_script_name; fastcgi_param PATH_INFO $document_uri; fastcgi_param PATH_TRANSLATED $document_root$fastcgi_path_info; fastcgi_param REQUEST_URI $request_uri; fastcgi_param DOCUMENT_URI $document_uri; fastcgi_param DOCUMENT_ROOT $document_root; fastcgi_param SERVER_PROTOCOL $server_protocol; fastcgi_param GATEWAY_INTERFACE CGI/1.1; fastcgi_param SERVER_SOFTWARE nginx/$nginx_version; fastcgi_param REMOTE_ADDR $remote_addr; fastcgi_param REMOTE_PORT $remote_port; fastcgi_param SERVER_ADDR $server_addr; fastcgi_param SERVER_PORT $server_port; fastcgi_param SERVER_NAME $server_name; fastcgi_param HTTPS $https; fastcgi_pass unix:/var/www/run/httpd.sock; } } .Ed .Pp Now, start the process using .Xr kfcgi 8 . It is recommended though, that the process lives in a clean chroot but in the example we will skip that because .Xr scid 8 cannot be built as static binary yet, you could as an alternative setup a chroot where all required libraries are available in order to run .Xr scid 8 inside of it. .Pp We will assume that the webserver is running with .Em www user and group which is the default also used with .Xr kfcgi 8 utility. The webserver must have read/write access to the UNIX socket generated with .Xr kfcgi 8 . .Pp This command needs to be ran as root but it will drop privileges to appropriate users and groups. See .Xr kfcgi 8 for more details. .Bd -literal -offset indent # kfcgi -p / -- scid -f Or # kfcgi -p / -u www -U www -- scid -f -d /path/to/sci.db -t /path/to/theme Or if you have a functional chroot (scid path is relative to it) # kfcgi -p /srv/sci -- /usr/bin/scid -f .Ed .Pp Make sure that .Xr scid 8 get read/write access to the default database path if you're not using the .Fl d option. Also don't forget the .Fl f option which indicates the process to run as FastCGI rather than plain CGI. .Pp If the command succeeded, Retrieve the stored key: .Bd -literal -offset indent $ scid api-get 1234567890secretABCDEF .Ed .Pp Both .Xr scictl 8 and .Xr sciworkerd 8 understand the same options and environment variables. So let's set the API key as environment variable for the next chapters. .Bd -literal -offset indent export SCI_API_KEY=1234567890secretABCDEF .Ed .Pp If you run .Xr scid 8 on a machine that is not on the same as the worker, you also need to specify the .Ev SCI_API_URL environment variable. .Bd -literal -offset indent export SCI_API_URL=http://127.0.0.1 .Ed .\" Setup a worker .Ss Setup a worker To register a worker, use the .Cm worker-add command from .Xr scictl 8 utility. .Bd -literal -offset indent scictl worker-add openbsd "OpenBSD 7.2" .Ed .Pp It is .Em strongly advised to run a .Xr sciworkerd 8 instance inside a chroot or a virtual machine, remember that it will fetch the script code remotely! .Bd -literal -offset indent $ sciworkerd Or more secure alternative # chroot /src/sci /usr/bin/sciworkerd -j2 .Ed .Pp Please make sure to read .Xr sciworkerd 8 manual page for more tuning options. .\" Register a project .Ss Register a project Create a project named .Em hello with a description, URL, a homepage and a script file to execute. The script code can be of any language but it's advised that you stick with some interpreted language as many different workers may execute it. .Bd -literal -offset indent $ scictl project-add hello "Hello World" "http://hello.org" "hello.sh" .Ed .\" Register jobs .Ss Register jobs Now, you may want to register a job that will be executed for every worker. .Pp Use the .Cm job-add command from .Xr scictl 8 utility. The .Ar tag argument is the unique argument that will be passed to the script code. In our case we will assume it's a revision from a SCM repository. .Bd -literal -offset indent $ scictl job-add hello 67470b67e460 .Ed .Pp And after that, any .Xr sciworkerd 8 instance will fetch the job, run it and send the result. It is best used with your SCM to add automatic job when pushing changes. .Pp Note: Jobs that are created before the registration of a worker won't be executed as it would create a high number of jobs to be performed each time you create a new worker. .\" SEE ALSO .Sh SEE ALSO .Xr scictl 8 , .Xr scid 8 , .Xr sciworkerd 8