setup.md (5347B)
1 amprolla setup 2 ============== 3 4 amprolla should be able to run on any system supporting Python 3, rsync 5 and GnuPG. However it has only been tested on Devuan and Gentoo. 6 7 8 Installation 9 ------------ 10 11 The recommended way to install Python 3 and the needded modules, along 12 with the extra needed dependencies is using your package manager. 13 14 You will need the following: 15 16 ``` 17 python3, python-requests, gnupg2, rsync 18 ``` 19 20 After installing the required dependencies, clone the amprolla git repo 21 using git to a place of your preference. 22 23 You will also need to setup a valid gnupg directory structure, along with 24 a key you shall use for signing the Release files if you require to do so. 25 26 27 Configuration 28 ------------- 29 30 To configure amprolla, a default configuration file is provided in 31 `lib/config.def.py`. Copy the file to `lib/config.py` and edit it to 32 your needs. The configuration file contains all the information needed 33 to properly merge the required repositories. The default configuration 34 is also working as long as you provide a valid gpg fingerprint used to 35 sign the Release files. If you do not wish to sign Release file, make 36 sure to disable it in the configuration file. 37 38 The `*dir` variables in the configuration file are the directories 39 where the files that are being merged are kept and the merges itself 40 are done. They can be either absolute or relative paths to the root 41 amprolla directory. The prefered way is to actually have absolute paths 42 as this will cause less trouble. 43 44 `banpkgs` is a set of package names that amprolla will refuse to merge 45 if they are found either in the dependencies of a package or if they 46 are the package itself. 47 48 `repo_order` is a list that holds that is ordered in the priority the 49 packages are prefered. The preference is ordered first to last. 50 The dict `repos` itself holds the required information for them. 51 52 ### repos dict structure 53 54 * The variable is a normal dict where the key is a string that should 55 be contained in `repo_order`. They key's values are another dict with 56 the following fields: 57 58 - `name`: the name of the repository used for rewriting the path to 59 the package's upstream (deb file). It is used in the nginx config 60 provided in the `contrib` directory. 61 62 - `host`: the upstream URL where the repository is being held. Used 63 to know from where to download the necessary Packages/Release files 64 and how to rewrite certain values. 65 66 - `dists`: the root directory where the suites are held. It is appended 67 to the above URL. 68 69 - `aliases`: if True, when downloading/rewriting, will look for the 70 suite's alias defined later in the configuration file. 71 72 - `skipmissing`: a hack helpful to skip missing suites if the repo 73 we are merging does not contain them. 74 75 76 `suites` hold our release names and their suites. 77 78 `aliases` hold the suites we want to merge as aliases in case we know a 79 certain repository doesn't use the same name for it. 80 81 `release_aliases` list our stable and testing branch aliases. 82 83 `categories` hold the package categories we wish to actually merge. apt 84 repositories usually hold `main`, `contrib`, and `non-free`. With this 85 we can opt out of any of them. 86 87 `arches` list the actual architectures we want to merge. If we are not 88 using certain architectures, it is easy to exclude them from merging 89 this way. 90 91 It is advised to not touch any variables listed below these, as they 92 are currently setup to provide a correct (valid) apt repository. 93 94 95 Running amprolla 96 ---------------- 97 98 After you've setup amprolla, it is needed to perform the initial full 99 download and full merge. First run `amprolla_init.py`, which is going 100 to download the necessary directory structures (as defined through the 101 config file) we will merge afterwards. When the download is done, it is 102 time to perform the full initial merge of these repositories. This will 103 provide us with a complete merged repository and we will then be able 104 to easily perform incremental updates of it. 105 106 After the first merge has been performed, it is advisable to run a 107 script called `populate_aliases.sh` found in the `contrib` directory. 108 Make sure you edit it and set a proper path to your merged directory, 109 and fill out the proper names needed. It will populate the merged 110 directory with symlinks to certain versions such as `1.0`, `stable`, or 111 `testing`. 112 113 To merge Contents files, run `amprolla_merge_contents.py`. This module 114 does not do incremental updates and should not be ran often due to its 115 heavy IO/RAM requirements. 116 117 Incremental updates are performed through `amprolla_update.py`, however, 118 for more stable performance and uptime, the incremental updating is 119 being orchestrated by a shell script called `orchestrate.sh`. This shell 120 script holds the logic to have near-atomic switching of repositories to 121 minimize repo downtime during performed merges. Not doing this could 122 result with users downloading corrupted repository files if they are 123 requested during an ongoing merge. 124 125 In a screen/tmux session, simply execute the `orchestrate.sh` script 126 and it will start looping and doing incremental updates every hour. 127 If you prefer, you can remove this loop and run the shell script through 128 a cron job based on your needs. 129 130 To actually serve the merged directory over HTTP, a basic nginx 131 configuration is provided as `contrib/nginx.conf`, and a lighttpd conf 132 is provided in `contrib/lighttpd.conf`.