Skip Navigation
Linux @lemmy.ml ladyanita22 @feddit.nl

Nix 3rd party repos and Binary packages

I've read a lot, but didn't get a simple answer for 3 topics I'm interested about:

  1. Are there any 3rd party repos for NixOS? It seems some people overlay one package, but I cannot find entire packagesets such as the one in Nixpkgs. Are channels used for that? Would I be able to have my own repo with several packages? Can I have more than 1 repo in my Nix system, and set which package install each program from?
  2. Would it be possible, for propietary software compiled directly for NixOS, to have a binary-only (as opposed to source, or binary+patched) repository? Everything I see is either source+cache or binary+patched, but nothing like binary especifically for NixOS.
  3. Let's say I want to add some extra, propietary codecs unavailable to NixOS. How would I be able to change a dependency so that each package dependent on it automatically picked it? Could I switch a dependency for a certain repo from one of another, different repo?

Thank you for your support guys!

12
12 comments

  • Are there any 3rd party repos for NixOS? It seems some people overlay one package, but I cannot find entire packagesets such as the one in Nixpkgs. Are channels used for that? Would I be able to have my own repo with several packages? Can I have more than 1 repo in my Nix system, and set which package install each program from?

    Yes, the most common one I think is the NUR, the Nix User Repository, the packages have their own namespace. https://github.com/nix-community/NUR

    Would it be possible, for propietary software compiled directly for NixOS, to have a binary-only (as opposed to source, or binary+patched) repository? Everything I see is either source+cache or binary+patched, but nothing like binary especifically for NixOS.

    I don't see anything prohibiting that

    Let’s say I want to add some extra, propietary codecs unavailable to NixOS. How would I be able to change a dependency so that each package dependent on it automatically picked it? Could I switch a dependency for a certain repo from one of another, different repo?

    The issue is that packages (or derivations) only depend on something because it's declared in their definition. That means if you need to add dependencies to existing packages, that can be done via overrideAttrs. However, I'm not certain that you can easily add attributes to functions that don't have a mechanism for it (I don't know derivations that allow for more attributes than specified and I'm not very good at the nix language) and dependencies (buildInputs and nativeBuildInputs) can only use packages that were supplied as attributes. Derivations usually declare their attributes via { x, y ? "bar" }:, what you would need to make use of additional attributes is something like attrs @ { x, y ? "bar", ... }: so that the additionally supplied attributes can be accessed. But again am not an expert in the language so take that with a grain of salt.

  • Are there any 3rd party repos for NixOS?

    Define "repository". For most definitions of a repository the answer would technically be "yes" but yours might differ.

    Would it be possible, for propietary software compiled directly for NixOS, to have a binary-only (as opposed to source, or binary+patched) repository?

    No. For a "native" Nix output path, you must have the full build definitions; Nix expressions.

    In the case of most proprietary software in Nixpkgs, the BLOB of the actual thing is a FOD (fixed-output-derivation) which obfuscates everything behind it; meaning it can't depend on other Nix output paths. Only the the dependency DAG for the wrapping around it is known.

    Let’s say I want to add some extra, propietary codecs unavailable to NixOS. How would I be able to change a dependency so that each package dependent on it automatically picked it?

    This is what overlays are for. They allow you to change the meaning of some package name in the context of an entire package set in which other packages may depend on the meaning of that package name. Those packages will have the change in meaning propagated to them.

    Note that changing a low-level package with many dependants such as i.e. mesa will cause a change in those dependants derivations; a change that is not present in the expression used for the central binary cache and therefore requires you to build the packages yourself. With a decently fast computer, building a typical desktop closure from such a low point onwards will take at least half a day.

    • No. For a “native” Nix output path, you must have the full build definitions; Nix expressions.

      I think I originally misunderstood the question. But I'm still not sure.

      This is what overlays are for. They allow you to change the meaning of some package name in the context of an entire package set in which other packages may depend on the meaning of that package name. Those packages will have the change in meaning propagated to them.

      The issue with how OP stated it though is that it basically means adding a dependency to a multitude of packages, not changing an existing one. I mean sure I could overlay ffmpeg-proprietary for ffmpeg and all other packages depending on ffmpeg would happily pick it up (as you said probably needing a rebuild) but how would you add my-proprietary-codec to all packages supporting it? Or how would you add JPEG-XL to ffmpeg and other packages that make use of it independently (e.g. so that Gwenview supports JPEG-XL)? I'm not saying it can't be done, but I personally don't know how. The latter might need a separate derivation for the plugin and a change in the derivation for Gwenview itself that can not be achieved through overrideAttrs for buildInputs alone. Or is my understanding incorrect?

      • The issue with how OP stated it though is that it basically means adding a dependency to a multitude of packages

        How would I be able to change a dependency so that each package dependent on it automatically picked it?

        how would you add my-proprietary-codec to all packages supporting it?

        Gather the list of attribute names and then define an overlay like this:

        final: prev: lib.genAttrs (n: prev."${n}".overrideAttrs (old: {
  buildInputs = old.buildInputs or [ ] ++ [ my-proprietary-codec ];
})) packageNamesToModify

        It should be said that packages usually don't work like this however (they usually depend on one central thing which then depends on some proprietary thing) and if they do, there's usually an override setting for the package in question with the dep already wired up.

        What you're actually looking for here is a Gentoo USE-flags like system. We have a precursor of that in i.e. config.cudaSupport but it's not widely used.

        how would you add JPEG-XL to ffmpeg

        You'd do that upstream because that's something that every instance of ffmpeg should be able to do. And also ping me because I maintain ffmpeg ;)

        and other packages that make use of it independently (e.g. so that Gwenview supports JPEG-XL)?

        I'd expect most packages to make use of such a feature dependently. I don't know how gwenview or QT stuff works but from a quick Google I gather that KDE stuff has the kimageformats library which then in turn depends on jxl. Gwenview should depend on that but only depends on qtimageformats currently? I'd give adding kimageformats to its buildInputs a spin if you're curious. If it works, make sure to submit a patch against Nixpkgs.

        This isn't the sort of thing you should solve in an overlay (better just do it upstream) but you certainly could.

  • Are there any 3rd party repos?

    Nix has "flakes", which allow you to share Nix code in a Git repository, it's like repos on steroids. There are many Git projects that offer new packages (such as nix-gaming) or NixOS modules (such as my project nixos-router), or even just Nix code (such as my projects notlua and notnft, which allow you to write Lua code and nftables rules in Nix), or any combination thereof.

    Would it be possible [for proprietary software to be compiled for NixOS]?

    Kind of. You first have to understand what Nix derivations are - builders that take certain inputs (such as certain versions of libraries) and produce some outputs.

    What happens if the inputs (such as a library version) change? The outputs change as well - previously it was /nix/store/abcdefgh-libfoo/lib/libfoo.so, now it's /nix/store/ijklmnop-libfoo/lib/libfoo.so - the path to libfoo changes, and the binary's RPATH reflects that.

    So if you want to package binary software for NixOS, you either have to pin library versions (so the paths don't ever change), or patch the binary.

    ...proprietary codecs...

    It depends on what those codecs are.

    Let's say they are a binary. In that case, you install them and they get added to your PATH - easy.

    Let's say they are some data files. In that case you install it and it gets put into XDG_DATA_DIRS - easy.

    Let's say it's a shared library (.so). First question - how is that .so loaded? By which program? From where?

    Depending on the answer, what you have to do changes as well. You may have to override some core media library, or ffmpeg, or maybe you can override VLC, or VLC's ffmpeg, but not system ffmpeg. Or, it may be the case that a simple LD_LIBRARY_PATH change will do it for you.

    Basically - it depends. That's why NixOS requires a deeper knowledge of Linux, or forces you to learn.

12 comments