I’ve been making another attempt to replace Docker with Podman. The issue is I can’t connect to my server through a web browser. I think it’s a firewall issue.

Networking and networking troubleshooting is a bit confusing for me and that’s the least favourite part about self hosting for me. Turns out I actually enjoy writing scripts more and the challenge of writing POSIX scripts especially.

If I can figure it out, I’ll probably write a guide for setting up Podman and Caddy on Alpine Linux since there isn’t a lot of recent information out there from what I found in my searches so far.

I use rsync for many of the reasons covered in the video. It’s widely available and has a long history. To me that feels important because it’s had time to become stable and reliable. Using Linux is a hobby for me so my needs are quite low. It’s nice to have a tool that just works.

I use it for all my backups and moving my backups to off network locations as well as file/folder transfers on my own network.

I even made my own tool (https://codeberg.org/taters/rTransfer) to simplify all my rsync commands into readable files because rsync commands can get quite long and overwhelming. It’s especially useful chaining multiple rsync commands together to run under a single command.

I’ve tried other backup and syncing programs and I’ve had bad experiences with all of them. Other backup programs have failed to restore my system. Syncing programs constantly stop working and I got tired of always troubleshooting. Rsync when set up properly has given me a lot less headaches.

Yeah, a few weeks ago a achieved my state of “secure” for my server. I just happened to notice a dramatic decrease in activity and that’s what prompted this question that’s been sitting in the back of my mind for weeks now.

I do think it’s important to talk about it though because there seems to be a lack of talk about security in general for self hosting. So many guides focus on getting services up and running as fast as possible but don’t give security much thought.

I just so happened to have gained an interest for the security aspect of self hosting over hosting actual services. My risks for self hosting is extremely low so I’ve reached a point of diminishing returns on security but the mind is still curious and wants to know more.

I might write up a guide/walkthrough of my setup in the future but that’s low priority. I have some other not self hosting related things I want to focus on first.

I think I am already doing that. My Kiwix docker container port is set to 127.0.0.1:8080:8080 and my reverse proxy is only open to port 12345 but will redirect kiwi.example. com:12345 to port 8080 on the local machine.

I’ve learned that docker likes to manipulate iptables without any notice to other programs like UFW. I have to be specific in making sure docker containers only announce themselves to the local machine only.

I’ve also used this guide to harden Caddy and adjusted that to my needs. I took the advice from another user and use wildcard domain certs instead of issuing certs for each sub domain, that way only the wildcard domain is visible when I search it up at https://crt.sh/ . That way I’m not advertising my sub domains that I am using.

I started with a local scan first, something like nmap 192 168.40.xxx for a specific device or nmap 192.168.40.0/24 for everything in your current network.

Nmap is quite complex with lots of options but there’s a lot of guides online to help out with the basics. You can press enter in your terminal while the scan is running and it should give a progress report.

My ISP blocks incoming data to common ports unless you get a business account. That’s why I used Cloudflare’s tunnel service initially. I changed my plans with the domain name I currently own and I don’t feel comfortable giving more power and data to an American Tech company so this is my alternative path.

I use Caddy as my reverse proxy so I only have one uncommon port open. My plans changed from many people accessing my site to just me and very few select friends of mine which does not need a business account.

I get that.

I was generally (in my head) speaking about all my devices. If someone stole my computer, the full disk encryption is more of a deterrence than the idea of my data being fully secured. My hope is that the third party is more likely to delete than to access. If I catch the attention of someone that actually wants my data, I have bigger issues to worry about than security of my electronic devices.

I agree with the last point, I only mentioned that because I don’t really know what other setting in my SSHD config is hiding my SSH port from nmap scans. That just happened to be the last change I remember doing before running an nmap scan again and finding my SSH port no longer showed up.

Accessing SSH still works as expected with my keys and for my use case, I don’t believe I need an additional passphrase. Self hosting is just a hobby for me and I am very intentional with what I place on my web facing server.

I want to be secure enough but I’m also very willing to unplug and walk away if I happen to catch unwanted attention.

Thanks for the insight. It’s useful to know what tools are out there and what they can do. I was only aware of nmap before which I use to make sure the only ports open are the ports I want open.

My web facing device only serves static sites and a file server with non identifiable data I feel indifferent about being on the internet. No databases or stress if it gets targeted or goes down.

Even then, I still like to know how things work. Technology today is built on so many layers of abstraction, it all feels like an infinite rabbit hole now. It’s hard to look at any piece of technology as secure these days.

I use a different port for SSH, I also have use authorized keys. My SSHD is setup to only accept keys with no passwords and no keyboard input. Also when I run nmap on my server, the SSH port does not show up. I’ve never been too sure how hidden the SSH port is beyond the nmap scan but just assumed it would be discovered somehow if someone was determined enough.

In the past month I did rename my devices and account names to things less obvious. I also took the suggestion from someone in this community and setup my TLS to use wildcard domain certs. That way my sub domains aren’t being advertised on the public list used by Certificate Authorities. I simply don’t use the base domain name anymore.

Early when I was learning self hosting, I lost my work and progress a lot. Through all that I learned how to make a really solid backup/restore system that works consistently.

Each device I own has it’s own local backup. I copy those backups to a partition on my computer dedicated to backups, and that partition gets copied again to an external SSD which can be disconnected. Restoring from external SSD to my Computer’s backup partition to each device all works to my liking. I feel quite confident with my setup. It took a lot of failure to gain that confidence.

I also spent time hardening my system. I went through this Linux hardening guide and applied what I thought would be appropriate for my web facing server. Since the guide seems more for a personal computer (I think), the majority of it didn’t apply to my use case. I also use Alpine Linux so there was even less I could do for my system but it was still helpful in understanding how much effort it is to secure a computer.

That’s been my main goal throughout securing my personal devices including my web facing server. To make things inconvenient as possible for potential outside interference. Even if it means simply wasting their time.

With how complex computers and other electronic devices have become, I never expect anything I own to be 100% secure even if I take steps I think will make me secure.

I’ve been on the internet long enough to have built a habit of obscuring my online or digital presence. It won’t save me but it makes me less or a target.

I found BashWrite which is just a very simple static site generator written completely in bash as a single file script.

The only dependency is having an up-to-date sed command which most systems should have. I use Alpine Linux which comes with a minimal sed command so I had to download the full command through my package manager.

It’s simple, basic and has support for the majority of markdown formatting. There’s some limitations due to it being written in Bash only but I am personally okay with that.

I found it on this list of static site generators if you’re curious to see more options.

It looks interesting and seems like it would be easy to set up. I’ll play with it and see how I like it. Thanks for the suggestion

Since my logs barely move, I just made aliases to where the logs are so it’s quick display and scan them within the terminal. I’m basically just viewing the system logs, fail2ban log and Caddy’s log so it’s fairly quick and simple for me.

The only change I’d like to do is change the output of Caddy’s log file so it’s not a long single line of information per output. I’ll have to do a bit more reading on that so I know what information I want to keep and how I want to visually organize it. At least for the moment, I am familiarising myself with what I am looking at and am slowly figuring out what information is relevant to me.

I like to keep my systems as simple and lean as possible which seems to strongly reflect my general approach to life. I find that kind of interesting.

I feel like my little Pi server is set up nicely now. At least I’m at the point where I’m not concerned about technically maintaining it. It’s as secure as I want it to be and I’ve tweaked my maintenance scripts slightly to avoid any unexpected issues.

I tried installing snikket but I couldn’t figure out how to get it to work with my Caddyfile using my current wildcard domain cert configuration. I’ll try again another time when I’m motivated again. It’s a low priority to me.

The last changes I made were adding logs and making them accessible to myself. So far they are all boring and predictable. Which is good news. It’s also nice to see that I’m the only person accessing it. The bots haven’t found my little corner of the internet yet.

Right now I’m taking a break from self-hosted stuff to work on my gardens and two artsy projects. A wooden carving for a friend’s birthday and an overly complicated shell script that has no real purpose. Although I’ve learned lots from it already so it’s not a complete waste of time.

I use rsync too. It’s older and from what I understand was designed at a time when data storage was much smaller so it may not be as fast as other backup options. It also doesn’t have encrypted backups like other backup options (I think).

Rsync has been the most reliable option for me though. Every syncing option I’ve tried seems too complicated and breaks down every time I look away. Since my entire backup size is around 550gb and I’m not concerned with encrypted backups, I think rsync just works just fine.

I even created my own tool that puts my rsync commands into easy to read/modify files so I can organize my most common transfers. I can easily backup my phone, HomeAssistant server, home server and computer to my two backup locations in a single alias or cronjob now.

A bit of a pain to learning how to make proper backups that restore successfully every time, but once I figured it out, I’ve been very confident in my backup strategy.

podman ps shows the following:

CONTAINER ID  IMAGE                                 COMMAND               CREATED         STATUS         PORTS                                                         NAMES
daae60bdcc65  docker.io/library/caddy-caddy:latest  caddy run --confi...  47 minutes ago  Up 47 minutes  0.0.0.0:80->80/tcp, 0.0.0.0:5050->443/tcp, 2019/tcp, 443/udp  caddy

netstat -tunpl shows the following:

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:5025            0.0.0.0:*               LISTEN      3270/sshd: /usr/sbi 
tcp        0      0 0.0.0.0:5050            0.0.0.0:*               LISTEN      7342/conmon         
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      7342/conmon         
tcp        0      0 10.89.0.1:53            0.0.0.0:*               LISTEN      7336/aardvark-dns   
tcp6       0      0 :::5025                 :::*                    LISTEN      3270/sshd: /usr/sbi 
udp        0      0 10.89.0.1:53            0.0.0.0:*                           7336/aardvark-dns 

The only difference for the netstat command between Docker and Podman is that Podman show’s entries for aardvark-dns and Docker does not which is something I expect.

I finally got Caddy’s TLS working with a custom module to handle DNS challenges. Turns out all I had to do was wait 10-15 minutes and everything would sort itself out.

Now on to the next puzzle. I started with Caddy in a Docker container and it’s working as intended. Now I want to replicate that in Rootful Podman Compose but I’m running into an issue. With the exact same setup (docker-compose.yml, Dockerfile and Caddyfile) I can get my TLS cert without issue but I can’t seem to connect to my website from any external browser. Not through my domain name or even through my home’s local network.

Once I figure out how I can access my website, I’ll be one step closer to where I want to be. Next will be to get Rootless Podman working, then I can finally set up the file server and kiwix instance instead of the test page I am currently using.

After that, I can finally spend time doing what I want to do and focus my time looking into the Gemeni Protocol.

Down the road I’ll look into hosting an IRC server and Snikket instant messenger but that’s super low priority. I like tinkering with my Raspberry Pi and my constant backup/restores wouldn’t be good for reliability for such services.

I sat down and managed to get wildcard certs working.

I figured I would leave my Caddyfile here in case anyone in the future needs a working reference. This is based off the Caddyfile mentioned in the original post.

Caddy Reference

# GLOBAL ENCRYPTION - DESEC.IO
{
        acme_dns desec {
                token "DeSEC.io Token Number"
        }
}

*.samplesite.ca {
        # SITE WIDE ENCRYPTION
        tls {
                dns desec {
                        token "DeSEC.io Token Number"
                }
        }
        # SUB DOMAIN #1
        @files host files.samplesite.ca
        handle @files {
                root * /srv
                file_server {
                        hide misc
                        browse
                }
        }
        # FALLBACK FOR UNHANDLED DOMAINS
        handle {
                abort
        }
}