Compiling s3backer on MacOS, 2025 edition

s3backer allows you mount a single large file split into smaller chunks on S3 as a local file, via FUSE. In contrast, s3fs-fuse mounts S3 objects as individual files. s3backer is useful for hosting disk images formatted with a native filesystem. Although Apple subsequently introduced the Sparse Bundle image format, which also internally splits the image into multiple files, some reported that Sparse Bundle is limited to 100,000 bands, with a band size > 8.4MB each. On the other hand, s3backer supports better local caching, more granular block sizes, and builtin compression and encryption.

This is a quick note about how to compile s3backer.

• If not already, install the Xcode command line tools: xcode-select --install
• I recommend using either MacPorts or HomeBrew to install a recent version of automake, autoconf, and (lib)curl.
• Install MacFUSE by downloading the appropriate package. This installs the header files under /usr/local/include and library files under /usr/local/lib.
• Clone s3backer: git clone https://github.com/archiecobbs/s3backer

The commands I use to build:

git clean -fxd
mkdir -p m4
autoreconf -iv

configure \
  --prefix=$HOME/local \
  CPPFLAGS='-DFUSE_DARWIN_ENABLE_EXTENSIONS=0 -I/usr/local/include/fuse3 -I/opt/local/include' \
  LDFLAGS='-L/usr/local/lib -L/opt/local/lib'

make -j8 install

Some explanations:

• git clean -fxd - removes any untracked files from the working dir.
• mkdir -p m4 - output files for autoreconf.
• autoreconf -iv - runs aclocal, automake, and autoconf.
• configure \
• --prefix=$HOME/local - pick your own install directory.
• CPPFLAGS
• -DFUSE_DARWIN_ENABLE_EXTENSIONS - on MacOS, fuse3 headers by default expects the code to use the Darwin file structs. This would have been useful for MacOS native filesystems, but s3backer uses the POSIX file structs.
• -I/usr/local/include/fuse3 - MacFUSE comes with both legacy fuse2 and the newer fuse3. If we included -I/usr/local/include, then #include <fuse.h> would pull in fuse2. This might have been useful for legacy code, but s3backer uses fuse3.
• -I/opt/local/include - add the MacPorts headers (for curl).
• LDFLAGS
• -I/usr/local/lib - MacFUSE library files.
• -I/opt/local/lib - MacPorts library files.
• make -j8 install - makes the s3backer binary and installs it to the --prefix configured above, using 8 parallel invocations.

It is worth noting that due to my specific setup, explained below, I ran into additional problems similar to this github issue: https://github.com/pyenv/pyenv/issues/3309, but I don't expect most people to run into it. I'll document my problem here for posterity.

During configure, gcc was not able to find the libcurl symbols. Upon closer inspection, gcc --version outputs x86_64-apple-darwin24.6.0 for the target, even though I'm now using an Apple Silicon machine, which should report arm64-apple-darwin24.6.0. The correct target was reported when I run gcc from the command line directly.

The reason for the wrong gcc target during configure is because I used a Python script to build s3backer out of tree. The script creates a unique temporary directory, cd into it, then runs the s3backer configure script from it. A long time ago, when I was still using an Intel mac, I installed Python from a python.org package. The x86 executables installed under /Library/Frameworks/Python.framework were migrated by the Migration Assistant when I bought a new Apple Silicon Mac.

When I run my Python script, it found the x86 Python interpreter, which is then run under Rosetta 2. Subsequently, when an x86 program runs gcc, it also runs the x86 version of gcc which would compile and link the output binary as x86. However, MacPorts only has arm64 of curl installed, which is why it couldn't find the curl symbols.

I'm not sure about the moral of the story. I would have preferred to start with a clean install, but it would have taken me too long to get it up and running. Migration Assistant gave me a usable machine more quickly but left some unexpected problems behind that needed clean up later. The technical debt trade-off was not known at that time, but an accidental one. I expect there will be more x86 files I need to clean up later.