And here I thought I was done with this madness.
repo commit: 0f601111962417711a73164bb861fb907ffcca7b
Recently, I logged into ^C once again, just to be met with a iris topic '556' (Geminispace blues), where a fellow ^Clubsmate decided to inquire about the state of our gemini capsules. I have reported that my scripts should still be generating content for both web and gemini, but as I was about to leave, an intrusive thought entered my mind.
"But what if plan9?"
But what if plan9 indeed? If gemini claims to be simple, why bother with a new protocol in the first place? We already have a simple and flexible way to transfer files from server to client; it's called 9p! But what about TOFU? 9p, if encrypted at all, uses centralised certificate authority! Well guess what; we already have that as well. It's called sftp and it's probably already running on your server without you even knowing about it. All you have to do is configure it a bit and here you go! Simple and efficient!
But you can't write custom sftp servers with special files, I hear you say. And yes, you can't easily do that. But do you know what you can do? Serve a FUSE filesystem.
And guess what! Since all file operations by a single connections are done by the same process, we can group all requests from one connection by their PID, even if the filesystem is mounted by sshfs and multiple commands are invoked over it. Cool!
So, you know, let's start simple. Just a little demo with one or two synthetic files, mount it somewhere into file tree and call it a day.
As you can see from my previous posts, I already have some experience with the fuse library for Go, so let's use that.
I start writing (copying code from one file to another) and the two things dawn on me.
First, this wasn't really an issue last time, as that was an extremely unusual filesystem, but structures used to implement all the 'Node' and 'Handle' interfaces need to be hashable, relying on a simple tree structure is not an option.
And second, more important revelation was that I can do better. The annoying part about writing a synthetic filesystem, be it FUSE or 9p, is that in addition to all the cool things you have in mind, you also have to write a file system. Who would want to replace gemini with that? What if instead, you just had your regular directory with your regular files, but any executable file, usually a script of some sorts, would be automagically transformed into a readable and writable synthetic file.
These script-files would either take the PID of the process handling them and generate some output to be displayed on read, or on write, they would take the PID and the written text and return the output on the next read, sort of like HTTP POST.
These scripts could do anything from simple "let's wrap fortune into markdown and complain on write" kinda deal, up to user accounts with sqlite database and separate process checking when the connection closes and users should therefor be logged out.
With a bit of hacking, I'm sure you could figure out some sort of form mechanism; truly magical indeed.
And so I started doing that instead I guess...
(how long is this b-log again? Oh, too much already. Nice.)
So, how do we represent the file hierarchy? I eventually settled on the following:
type Fid uint64
type Pid uint32
type FileNode struct {
Dirent fuse.Dirent;
FullPath string
IsExecutable bool
Size int64 // if not executable...
// needs to be pointer, as cannot take reference to map item
// alternative would be havind fileMap `map[Fid]*FileNode`, but that would be
// even more annoying
Children *[]Fid;
}
var fileMap map[Fid]FileNode
var rootFid Fid = 1
var nextFid = rootFid
'fileMap' maps FIDs to 'FileNode's, which also hold their FID as part of 'Dirent'. For simplicity, I just have one universal node for both files, and directories. Each node can then hold slice of FIDs of it's children. The problematic interfaces are then bound to the FIDs themselves. For non-executable files, I also get their length in advance. I guess this also means, that you should restart the server after every change made to them.
As for the discovery itself, I think that the only noteworthy part is that getting information about whether a file is executable to the specific process is a bit of a pain, so I left it on go-fileperm library.
As already mentioned, one of the nice parts of the Go type system that the fuse library utilizes is that you can bind the methods to whatever you like, which in my case is the FID. I also obtain the PID of requesting process like so:
func withContext(ctx context.Context, req fuse.Request) context.Context {
return context.WithValue(ctx, "PID", req.Hdr().Pid)
}
// and on fuse startup
server := fs.New(con, &fs.Config {
Debug: fuse.Debug, // do nothing
WithContext: withContext, // store PID in context
})
Startup itself is done in a rather specific way to allow for cleanly terminating the process with ^C.
// mount fuse
con, err := fuse.Mount(
mountpoint,
fuse.FSName("fuse/SFTH-api-test"),
fuse.Subtype("apifs"),
fuse.AllowOther(),
)
if err != nil { log.Fatal("cannot mount") }
// Set up signal handling for graceful shutdown
sigChan := make(chan os.Signal, 1)
sig nal.Notify(sigChan, os.Interrupt, syscall.SIGTERM)
// start serving the fs
serveDone := make(chan error, 1)
go func() {
server := fs.New(con, &fs.Config {
Debug: fuse.Debug, // do nothing
WithContext: withContext, // store PID in context
})
serveDone <- server.Serve(FS{})
}()
fmt.Println("Serving at", mountpoint)
select {
case <-serveDone:
case <-sigChan:
}
err = fuse.Unmount(mountpoint)
con.Close()
Reading non-executable files is quite straight forward. Reading executable files is a bit harder, as I don't know their size, until I execute them, so I report it as 0. This doesn't sit well with the kernel of all places, however, so I need to tell it to chill out a bit by setting
resp.Flags |= fuse.OpenDirectIO
in 'Open'.
As for writing, since UNIX file theory has it's origins far before files could just fit into RAM, it will require some buffering. Introducing, the cache:
type CacheEntry struct {
InputBuffer *[]byte
OutputBuffer []byte
}
var cache map[Pid]map[Fid]CacheEntry
func initCache() {
cache = make(map[Pid]map[Fid]CacheEntry)
}
// remove entries by dead processes
func CleanCache() {
for {
// clear cache every five minutes
time.Sleep(time.Minute * 5)
for k := range cache {
if !PidExists(k) {
delete(cache, k)
}
}
}
}
Yep, that's it. Nothing complex to see here. Totally not at all. But how do I check if 'PidExists' you might ask...
// dafaq! what is wrong with you, UNIX? U ok?
// Source - https://stackoverflow.com/a/59459658
// Posted by Paul
// Retrieved 2026-09-14, License - CC BY-SA 4.0
func PidExists(pidPropa Pid) bool {
pid := int32(pidPropa)
if pid <= 0 {
return false
}
proc, err := os.FindProcess(int(pid))
if err != nil {
return false
}
err = proc.Signal(syscall.Signal(0))
if err == nil {
return true
}
if err.Error() == "os: process already finished" {
return false
}
errno, ok := err.(syscall.Errno)
if !ok {
return false
}
switch errno {
case syscall.ESRCH:
return false
case syscall.EPERM:
return true
}
return false
}
But yea, that's about that. On 'Flush' I send the string to the command and store it's output in the other cache buffer to be returned, once the user decides to read it.
So now we can read, we can write... What's next?
So let's pretend that you want to deploy your very own 'script-serve-fs'. First thing you will need is a server to deploy on (no, you can't deploy on ^C).
On said sever, you will need an OpenSSH server running, which you most likely already have.
Next, you want to decide on some place for the filesystem. In our case, it will be '/var/sftp/source' for the source dir and '/var/sftp/export' for the mountpoint and the script itself will be placed next to it in '/var/sftp/script-serve-fs'.
You will want to have your FUSE filesystem to get mounted on boot via an init system of sorts. As my system runs on systemd, I will create a '/etc/systemd/system/script-serve-fs.service' with the following contents and enable it:
[Unit] Description=serves script filesystemly DefaultDependencies=no Wants=network-pre.target systemd-modules-load.service local-fs.target [Service] Type=simple ExecStart=/var/sftp/script-serve-fs /var/sftp/source /var/sftp/export ExecStop=/usr/bin/fusermount -u /var/sftp/export [Install] WantedBy=multi-user.target
Next, you will want a user of some kind to be the vessel for your sftp shenanigans. In our case, 'anon'.
# make the user sudo adduser anon # take away thier shelling priviledges sudo chsh -s /sbin/nologin anon # and their password as well sudo passwd -d anon # not they can't to shit
For my next trick, I will make their ssh rights disappear. Add the following to your '/etc/ssh/sshd_config':
Match User anon ChrootDirectory /var/sftp/export ForceCommand internal-sftp AllowTcpForwarding no X11Forwarding no PermitEmptyPasswords yes
This will allow anon to only use sftp and to only see contents of '/var/sftp/export'. Since anon doesn't even have a password anymore, users won't even be prompted for one.
And congratulatoins! You can now use your new 'script-serve-fs'!
Unless...
Here are some problems I faced:
Add 'fuse.AllowOther()' to 'fuse.Mount' and allow 'user_allow_other' in '/etc/fuse.conf'.
OpenSSH was build by crazed people and is very dependent on permissions. Your chroot target can't allow other that owner to write the directory.
You must use the '-o direct_io' flag to tell your system to chill out as well like so:
sshfs -o direct_io anon@unit37.duckdns.org: <mountpoint>
Your filesystem needs to implement 'Fsync'. It is basically 'Flush', but more, so they can to the same thing.
So why have I done this? Because I was bored I think... Probably.
feel free to ssh me @
sshfs -o direct_io anon@unit37.duckdns.org: <mountpoint>
or something, IDK