March 3, 2020

Setting up Git using SourceHut

In this post i will document the process of setting up selfhosted git. I decided to go with SourceHut because of it's modular approach making it less resource intensive. There install documentation leaves much to the user so this should be a fun interesting process. I'm going to try to keep this post organized but it might end up all over the place if so i apologize.

SourceHut is broken up into many modules. For my current needs i will only be setting up the core,meta(login) and git modules. More might be added in the future. So lets get started.

SourceHut provides a repository for Archlinux so step 1 is to add that. This is done by editing the /etc/pacman.conf file and adding this

[sr.ht]
Server = https://mirror.sr.ht/archlinux/sr.ht

after that packages can be installed normally

sudo pacman -Sy meta.sr.ht

This will pull in all required dependencies.

Now the fun begins. The SourceHut documentation states that every module comes with a config.example.ini but for the life of me i could not find it so i manually created the path/file /etc/sr.ht/config.ini and got the content from config.example.ini and carried on.

The config file itself in fairly well explained so i wont go into details here. Simply adapt to my needs. A couple options i'm unsure of

site-info sounds like a landing page type which i don't intend on having. Not sure if it can be blank at this point.

privacy-policy because this will pretty much be a personal/private setup i don't plan on having one of these. It's blank by default so i will leave it.

Next is the database related things. SourceHut uses postgresql as a database so first thing is to create a db and user. I use webmin to do most database tasks so i created the sourcehut user and the db metasrht making the sourcehut user it's owner . Then set connection-string property to

connection-string=postgresql://sourcehut@localhost/metasrht

so the create db function can do it's thing in the next step.

python3
>>> from metasrht.app import db
>>> db.create()

This will create the database for the meta module.

Next we should be able to start the meta.sr.ht service I had to make a change at this point to the port used. Making the change both in the systemd service(/usr/lib/systemd/system/meta.sr.ht.service) file and in the module config. This was required because of everything else i run on this server the port was already used.

Once that change was made all that was required to start the service was

sudo systemctl daemon-reload then sudo systemctl start meta.sr.ht sudo systemctl enable meta.sr.ht

So now the meta service is running but can't be accessed as it's running only locally. What we need is to setup a (sub)domain to point to our server then setup webserver(apahe in my case) to serve the application. So my apache config looks like this

<VirtualHost *:80>
   ServerName shmeta.marcg.pizza

   # Default is to force https
   RewriteEngine on
   RewriteCond %{SERVER_NAME} =shmeta.marcg.pizza
   RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,QSA,R=permanent]

   <Location "/.well-known/acme-challenge/">
      Options None
      Require all granted
   </Location>
</VirtualHost>


<IfModule mod_ssl.c>
<VirtualHost *:443>
   ServerName shmeta.marcg.pizza

   # Path to ErrorLog and access log
   ErrorLog ${APACHE_LOG_DIR}/shmeta.error.log
   CustomLog ${APACHE_LOG_DIR}/shmeta.access.log combined

   # TLS
   # Feel free to use your own configuration for SSL here or simply remove the
   # lines and move the configuration to the previous server block if you
   # don't want to run funkwhale behind https (this is not recommended)
   # have a look here for let's encrypt configuration:
   # https://certbot.eff.org/lets-encrypt/debianstretch-apache.html
   SSLEngine on
   SSLProxyEngine On
   SSLCertificateFile /etc/letsencrypt/live/marcg.pizza/fullchain.pem
   SSLCertificateKeyFile /etc/letsencrypt/live/marcg.pizza/privkey.pem


   

   # Configure Proxy settings
   # ProxyPreserveHost pass the original Host header to the backend server
   ProxyVia On
   ProxyPreserveHost On
   <IfModule mod_remoteip.c>
      RemoteIPHeader X-Forwarded-For
   </IfModule>

   # Turning ProxyRequests on and allowing proxying from all may allow
   # spammers to use your proxy to send email.
   ProxyRequests Off

   <Proxy *>
      AddDefaultCharset off
      Order Allow,Deny
      Allow from all
   </Proxy>

   <Location "/">
      LimitRequestBody 104857600

      Header set X-Frame-Options "sameorigin"
      Header set Content-Security-Policy "default-src 'self'; script-src 'self'; style-src 'self' 'unsafe-inline'; img-src 'self' data:; font-src 'self' data:; object-src 'none'; media-src 'self' data:"
      Header set Referrer-Policy "strict-origin-when-cross-origin"
      ProxyPass http://127.0.0.1:5002/
      ProxyPassReverse http://127.0.0.1:5002/
   </Location>

   <Location "/static">
      ProxyPass  "!"
   </Location>
   Alias /static /usr/lib/python3.8/site-packages/metasrht/static
  <Directory /usr/lib/python3.8/site-packages/metasrht/static>
   Require all granted
  </Directory>
</VirtualHost>
</IfModule>

This is probably not the best config as i'm no expert here but it works. It's adapted off the config i use for funkwhale.

So great now we can get to the login page. Next part creating the initial admin user.

The documentation has some step on converting an existing user into an admin but nothing specific on creating the user. I saw 2 options

Turn on registration, create user,turn off registration,make user admin
Use the metasrht-createuser script i saw in the source to create the user

Problem was for some reason that script was not installed with the package ( it should be now ). So what i did was simply create a file and past the content of the script into it. Then execute it like so

python metasrht-createuser.py -t admin <user_name> <user_email>

This will prompt for a password and create the user. To prepare the DB for future migrations/upgrades do

srht-migrate meta.sr.ht stamp head && metasrht-migrate stamp head

Voila now we can login to the meta module.

I noticed another issue before moving on. The audit log was always showing 127.0.0.1 as the source IP on all entries. After much searching and trial and error i found a working solution. I sent a patch for the meta package. Not sure if it will get merged as it may not be the ideal way to do this. It works for me.

Moving on to git.

The git.sr.ht is again installed like any other package

sudo pacman -S git.sr.ht this will install it and it's dependencies

Now we look at config.example.ini and add the [git.sr.ht] and [git.sr.ht::dispatch] sections to our /etc/sr.ht/config.ini making the require changes and setting up oauth from our meta installation.

Now setup the DB

python3
>>> from gitsrht.app import db
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.8/site-packages/gitsrht/app.py", line 7, in <module>
    from gitsrht.repos import GitRepoApi
  File "/usr/lib/python3.8/site-packages/gitsrht/repos.py", line 5, in <module>
    from minio import Minio
  File "/usr/lib/python3.8/site-packages/minio/__init__.py", line 36, in <module>
    from .api import Minio
  File "/usr/lib/python3.8/site-packages/minio/api.py", line 49, in <module>
    import certifi
ModuleNotFoundError: No module named 'certifi'

What's this? A missing dependancie.(I've sent a patch to add it to the package so this may no longer be an issues) No problem lets install it

sudo pacman -S python-certifi

then again

python3
>>> from metasrht.app import db
>>> db.create()

This gave a warning “Unable to ensure delegated scopes are provisioned. Is https://shmeta.marcg.pizza reachable? This may render the API unusable.” Lets see if everything works in the end.

Next prepare the DB for future migrations

srht-migrate git.sr.ht stamp head && gitsrht-migrate stamp head

Setup the folder to hold repos

sudo mkdir /srv/gitrepos or whatever you set in your config

and make the gitsrht user it's owner

sudo chown -R gitsrht:gitsrht /srv/gitrepos/

Now the apache config. Just copy the one for the meta module and adapt the servername and such.

Make sure to add to apache main config and restart apache. Start the git.sr.ht service

Create cron entry or systemd timer to run /usr/bin/gitsrht-periodic every 20 minutes.

Example systemd timer/service

gitsrht-periodic.timer

[Unit]
Description=Run gitsrht-periodic every 20 minutes

[Timer]
OnBootSec=20min
OnUnitActiveSec=20min
Unit=gitsrht-periodic.service

[Install]
WantedBy=timers.target

gitsrht-periodic.service

[Unit]
Description=Nextcloud cron.php job

[Service]
User=gitsrht
ExecStart=/usr/bin/gitsrht-periodic

[Install]
WantedBy=basic.target

Then start/enable the timer.

This also needs to be done for metasrht-daily but have it run daily.

So with all this i now have a working meta module to handle login/users and a “working” git module. I have working in quotations here because altho i can login and create a repo i can't pull/push anything over ssh(did not setup/try over http(s)). I get some errors. Reach out for support i will post an update when i have it working.

And with that

That's all for now. Hope this post made sense. Let me know if you have any questions/comments.

See Part 2 where i get GIT fully working.

#tech #selfhost

Until next time. Stay safe!

G @mgrondin@youdabomb.social