Ordered list of “basic” skills that need to be acquired to enjoy the ride.
Applies to Linux, whatever the distribution, although we use NixOS ourselves and have a section dedicated to it.
Go through this slowly. It is tempting to speed-read through a book, call it done, and move on to the next book.
To get the most out of this, take your time understanding each section.
For every 5 minutes you spend reading you should spend 15 minutes tinkering around with what you just read. Play around, copy/past, break things, have fun.
A good general cheat sheet page: https://github.com/ruanbekker/cheatsheets#readme
Good read on how Unix started and influenced Linux: [https://www.redhat.com/sysadmin/unix-linux-history]
A shell is a user interface for access to an operating system’s services.
A terminal is a program that opens a graphical window and lets you interact with the shell.
It is common that the keyboard layout the system is configured with is different from the keyboard you actually use (e.g. system keyboard configured US layout but you use a keyboard with a French layout), read this article “https://www.baeldung.com/linux/console-change-keyboard-layout” to fix this.
A CLI (command-line interface) is what deal with when you interact with the shell.
The most important command is man
which stands for “manual”. It explains what the command is and how to use it. Don’t spend hours on youtube, read the manuals. e.g.
$ man git
NAME
git - the stupid content tracker
SYNOPSIS
git [-v | --version] [-h | --help] [-C <path>] [-c <name>=<value>]
[--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]
[-p|--paginate|-P|--no-pager] [--no-replace-objects] [--bare]
[--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]
[--config-env=<name>=<envvar>] <command> [<args>]
DESCRIPTION
Git is a fast, scalable, distributed revision control system with an unusually rich command set that provides both high-level operations and full access to internals.
[...]
h
for help and q
to leave the manual./
e.g. /ignore
then hit:
n
for next matchShift + n
for previous matchapropos
.cat /etc/os-release
hostnamectl
Cmd
+ Alt
+ =
.Cmd
+ Enter
.Cmd
+ +
.Cmd
+ -
.clear
.reset
.bash
and its history (sh, csh, tsh, ksh …).zsh
pronounced Z shell, is bash
on steroids with extended features and support for plugins and themes, most noticeably:
cd
.Environment variables in Linux-based systems:
$ printenv
: Print all the environment variables.
$SHELL
: The default shell being used on the system (e.x. zsh
, bash
…).$PATH
: Instructs the shell which directories to search for executables, it allows to run commands without having to specify its full path.$ export EDITOR=vi
$ echo $EDITOR
man
: User manual of given command.apropos
: Search all the man pages using keywords to find commands and their functions (read this).whatis
: Display manual documentation pages in various ways.pwd
, ls
, cd
, type
, mkdir
, mv
, rm
, ln
, which
, stat
, whereis
, cat
, head
, tail
, more
, tee
…uname -a
, hostname
, whoami
, passwd
, chown
, chgrp
, chmod
, …uptime
: Tell how long the system has been running.ip a
: Tells you the IP address of your system.su
/sudo
, doas
: Used to assume the identity of another user on the system (they are both similar tools, doas
has been ported from the OpenBSD project and could be assumed “safer” than sudo
as it is less error-prone e.g. when setting up somewhat complicated patterns in /etc/sudoers
).history
echo "$HISTFILE"
history | grep [string]
: Find any record in history.history -c
: Remove all records.history -d 1234
: Remove record number 1234.
fc
which doesn’t have an option to delete records from the history. <workaround>Ctrl
+ s
and Ctrl
+ r
respectively; Read this if ‘i-search’ using Ctrl
+ s
does not work.alias
grep
:
$ grep 'keyword' /path/to/file.log
$ grep -B 5 -A 2 --color 'keyword' /path/to/file.log
grep -v "pattern" file
:
# journalctl -n 50 |grep -v "sudo"
# journalctl -n 50 |grep -v -e "sudo" -e "background"
grep -rnw '/path/to/somewhere/' -e 'pattern'
.c
or .h
extensions:
grep --include=\*.{c,h} -rnw '/path/to/somewhere/' -e "pattern"
.o
extension:
grep --exclude=\*.o -rnw '/path/to/somewhere/' -e "pattern"
--exclude-dir
parameter. For example, this will exclude the dirs dir1/
, dir2/
and all of them matching *.dst
:
grep --exclude-dir={dir1,dir2,*.dst} -rnw '/path/to/search/' -e "pattern"
tee
: a command in command-line interpreters using standard streams which reads standard input and writes it to both standard output and one or more files, effectively duplicating its input. It is primarily used in conjunction with pipes and filters. The command is named after the T-splitter used in plumbing.watch
: Execute a program periodically showing output in fullscreen e.g.$ watch 'du -ac -d0 /data/bitcoind/blocks'
$ watch -n 2 'bitcoin-cli -getinfo | grep progress'
md5sum
: Calculates and verifies 128-bit MD5 hashes as a compact digital fingerprint of a file. There is theoretically an unlimited number of files that will have any given MD5 hash.sha256
: Similar to md5 but based on 256-bits, considered more secure and less prone to theoretical collisions.find /search/directory/ -name "matching file search criteria" -actions
find /dir/to/search -name "pattern" -print
find /dir/to/search -name "file-to-search" -print
find /dir/to/search -name "file-to-search" -print [-action]
“If your work is not on Github it does not exist”
-Ben Arc
GitHub (or GitLab or other alternative) is a platform on the Internet used for storing, tracking, and collaborating on projects. It makes it easy to share all sorts of files and collaborate with peers in an asynchronous but coordinated way. GitHub also serves as a social networking site where developers can openly network, collaborate, and pitch their work.
$ man git
What is stash
/pop
, when to use them: https://www.youtube.com/watch?v=urSlkC-6lZE.
$ git config --global user.name "John Doe"
$ git config --global user.email johndoe@example.com
more ~/.gitconfig
git config -l
git config user.name
git clone https://github.com/mybonk/mybonk-core.git
git remote show origin
git remote set-url
: You may need to change the connection type to GitHub from https to ssh (if you use private-public keys to connect). This is done by using the git remote set-url
and changing https://github.com/mybonk/mybonk-core.git
by git@github.com://github.com/mybonk/mybonk-core.git
.
$ git remote set-url origin git@github.com://github.com/mybonk/mybonk-core.git
git status
git show
git log main --graph
git branch
git switch
git checkout origin/main
/ git checkout origin/main <file_name>
git add .
/ git add -a
/ git add -A
git mv filename dir/filename
git restore --staged file-to-unstage.txt
: Unstage a file in the Git index and undo a git add operation. Command introduced in 2019 as (Git 2.23). Are NOT recommended git rm
(may remove the file not only from the index, but also from the repo altogether) and git reset
(is “too powerful” and may result in rewriting of branch history).
git commit -m "commit message"
git push
/ git push -u origin main
git pull
git reflog
git remote set-url origin https://git-repo/new-repository.git
/
git remote set-url <remote_name> <ssh_remote_url>
/
git remote -v
git diff <dir or file name>
git log
/ git log --oneline
/ git log -S "signet" --pretty=format:'%h %an %ad %s'
git blame README.md
(also try the options -e
, -w
, -M
, -C
, -L 1,5
)
git stash push
/ git stash list
/ git stash pop
/ git stash apply
Using one of the following might saves you time by simplifying your interactions with Git.
Ungit: “The easiest way to use git. On any platform. Anywhere.”
Command-line tool that brings pull requests, issues, GitHub Actions, and other GitHub features to your terminal, so you can do all your work in one place.
Authenticate with your GitHub account:
$ gh auth login
vi
(cheat-sheet HERE)
$ vi +132 myfile
: Open myfile on line 132sed
(https://www.gnu.org/software/sed/manual/sed.html): “stream editor” for editing streams of text too large to edit as a single file, or that might be generated on the fly as part of a larger data processing step: Substitution, replacing one block of text with another.awk
(https://github.com/onetrueawk/awk/blob/master/README.md): Programming language. Unlike many conventional languages, awk is “data driven”: you specify what kind of data you are interested in and the operations to be performed when that data is found.ripgrep
): Recursively search the current directory for lines matching a pattern, very useful to find whatever document containing whatever text in whatever [sub]directory.
$ rg what_i_am_looking_for MyDoc_a.php MyDoc_b.php
Look for string ‘what_i_am_looking_for’ in MyDoc_a.php and MyDoc_b.php$ rg what_i_am_looking_for MyDoc.php -C 2
: Return results with context, displaying 2 lines before and 2 lines after the match in the output. Also try the options -B
and -A
(number of lines before and after the match).$ rg 'Error|Exception' MyDoc.php
: Searches the file for either Error
or Exception
.$ rg 't.p' MyDoc.php
: Looks for a t
and a p
with any single character in between.$ rg ssl -i
: Recursively searches for instances of ssl in a case-insensitive manner.$ rg service /etc/var/configuration.nix -i
: Recursively searches a specific directory for instances of service in a case-insensitive manner.rg -g 'comp*' key
Searches only for the pattern key
in files beginning with the substring comp
.-l
option to only list the files having a match without additional details/-i
option for case-insensitive search.-S
option for smart case search.-F
option to treat the search string as a string literal (regex syntax).$ rg --type-list
List of the available file types.$ rg key -t json
Restricts the search for the pattern key to json files only.README.md
so that it is nicely formatted and readable on a terminal:
$ pandoc -t plain README.md | less
$ tar -xvf myfile.tar.gz
: Untar a file.lsblk
: List information about the system’s available or the specified block devices (e.g. disks, USB sticks …).hdparm -t --direct /dev/sda1
(if not installed run nix-shell -p hdparm).df
: “Disk Free” displays disk usage:
df -hT
. -h
for “human readable”, add -T
to display the type of the filesystem.df -hT .
. ‘.
’ for whatever partition the current directory is residing on.df -hT -t ext4
. -t ext4
to display only filesystems of type ext4.df -hT -x squashfs -x overlay -x tmpfs -x devtmpfs
to hide given filesystem types from the output.du
: “Disk Usage” to get space usage of directory/subdirectory.
$ du -h /data
. -h
for “human readable”.$ du -s /data
. -s
for “summary”.$ du -ah --time
. Shows the time of the last modification to any file in the directory or subdirectory. --time
with the -ah
flags is very useful e.g. someone writes files somewhere by accident and you need to find where.fdisk
: Dialog-driven program to see and manipulate disk partition table:
fdisk -l
: List the system’s partition scheme.fdisk -l | grep "Disk /"
: See all the available disks.fdisk /dev/sdc
: Enter the interactive mode to manipulate the partition table of the disk /dev/sdc
mkfs.ext4 /dev/sdc1
: Formats the partition /dev/sdc1
in ext4 file system format.
Great tutorial on how to use cURL: https://curl.se/docs/httpscripting.html
$ curl -O https://testdomain.com/testfile.tar.gz
: Download the file testfile.tar.gz
.$ curl -o mydownload.tar.gz https://testdomain.com/testfile.tar.gz
(little ‘o’): Download to the file mydownload.tar.gz
.$ curl -I https://www.google.com
: Query the header of a server (same as -head
).$ curl -k https://localhost/my_test_endpoint
: Ignore invalid or self-signed certificates (same as --insecure
).$ curl --data "param1=test1¶m2=test2" http://test.com
$ curl -H 'Content-Type: application/json' --data '{"param1":"test1","param2":"test2"}' http://www.test.com
$ curl -X 'PUT' -d '{"param1":"test1" "param2":"test3"}' \http://test.com/1
: Specify a type of request (here ‘PUT’).$ curl -u <user:password> https://my-test-api.com/endpoint1
: Basic Authentication for various protocols (same as -user
)./etc/hosts
host-resolution.$ curl -F @field_name=@path/to/local_file <upload_URL>
$ curl -F @field_name=@path/to/local_file_1, @field_name=@path/to/local_file_2, @field_name=@path/to/local_file_3, <upload_URL>
$ curl -w "%{time_total}\n" -o /dev/null -s www.test.com
: Use -w
to display information (stdout) after a transfer. total_time
is one of the interesting parameters curl returns$ wget --no-parent -r http://WEBSITE.com/DIRECTORY
Spare yourself the pain, learn good habits, save tones time and avoid getting locked out of your system by really understanding how SSH (a.k.a Secure Shell or Secure Socket Shell) works, how it allows you to connect a remote machine, execute commands, upload and download files.
~/.ssh/config
)ssh-keygen
e.x. $ ssh-keygen -t ecdsa -b 521
, ssh-keygen -R 5.72.114.57
passphrase
ssh-copy-id
: Copy your public key on the server machine’s ~/.ssh/authorized_keys
ssh-add
Use ssh auto login (auto login using public and private keys pair to be specific) as it is also significantly more secure than basic password-based login. Bellow is a real time illustration of ssh failed login attempts initiated from the Internet (bots, hackers, you name it) on a machine with password authentication left enabled (instead of using ssh auto login).
For reference:
$ssh root@192.168.0.155
#
IP addresses (here 192.168.0.155
) are not “human friendly”. You can associate an IP address to an arbitrary name, easier to remember. This can be configured in your ssh configuration file ~/.ssh/config
. Here is an example in its simplest form:
Host console_jay
HostName 192.168.0.155
User root
You can now use the following simple syntax instead to connect:
$ssh console_jay
This is all very nice until you change environment or move your hardware to another network: A new IP address will be assigned to the machine and the shorthand console_jay
will no longer work, you now have to figure out what the new IP address of your machine (scan the network or physically connect to serial) which is frustrating and time consuming.
A more effective way to deal with this issue and streamline remote access altogether is to use WireGuard/Tailscale. Tailscale basically hides all the nitty gritty of ssh by managing the VPN for you in the background.
Have a look at this Tailscale Quick tutorial.
To enable Tailscale:
With tailscale on you can now refer to your remote machine anytime anywhere through its Tailscale “Magic DNS” name:
$ tailscale ssh console_jay@dab-dominant.ts.net
Or even just:
$ tailscale ssh console_jay
Note that Tailscale’s “Magic DNS” and ssh commands are transparently wrapped through the Tailscale VPN. So $ ssh console_jay
will work as well as $ tailscale ssh console_jay
.
Commonly used:
# tailscaled
$ tailscale help
$ tailscale login
$ tailscale up
$ tailscale down
$ tailscale status
$ tailscale netcheck
$ tailscale ssh console_jay
rsync uses a delta transfer algorithm and a few optimizations to make the operation a lot faster compared to scp
. The files that have been copied already won’t be transferred again (unless they changed since). Can be run ad-hoc on the command line or configured to run as a deamon on the systems to keep files in sync.
rsync allows to restart failed transfers - you just reissue the same command and it will pick up where it left off, whereas scp will start again from scratch.
rsync needs to be used over SSH to be secure:
Example 1: From local to local (instead of using scp
):
$ rsync -avhW --progress --exclude --exclude '*/*.lock' /unmountme/bitcoind/{blocks,chainstate,indexes} /data/bitcoind
$ rsync -avhW --stats --exclude '*/*.lock' --human-readable /unmountme/bitcoind/{blocks,chainstate,indexes} /data/bitcoind | pv -lep -s $(find /unmountme/bitcoind/{chainstate,blocks,indexes} -type f | wc -l)
### Speed test
findssh: Super quick command line tool that scans an entire IPv4 subnet in less than 1 second. Without NMAP. It is extremely quick but sometimes it misses some hosts so run it a couple of time to be sure it scanned them all.
Example to scan the complete network for sshd listening:
$ python3 -m findssh -b 192.168.0.1 -s ssh -v
searching 192.168.0.0/24
DEBUG:asyncio:Using selector: EpollSelector
DEBUG:root:[Errno 111] Connect call failed ('192.168.0.19', 22)
(IPv4Address('192.168.0.82'), 'SSH-2.0-OpenSSH_8.4p1 Debian-5+d')
(IPv4Address('192.168.0.136'), 'SSH-2.0-OpenSSH_9.1')
(IPv4Address('192.168.0.106'), 'SSH-2.0-OpenSSH_8.4p1 Debian-5+d')
DEBUG:root:[Errno 111] Connect call failed ('192.168.0.150', 22)
(IPv4Address('192.168.0.100'), 'SSH-2.0-OpenSSH_7.4')
DEBUG:root:[Errno 111] Connect call failed ('192.168.0.44', 22)
### Tor
### I2P
… or alternatives like GNU Screen, Terminator, Byobu, .etc…
$tmux source-file ~/.tmux.conf
$tmux new -s MY_SESSION
$tmux list-keys
$tmux list-sessions
/ $tmux ls
$tmux kill-session -t name_of_session_to_kill
: Kills a specific session.
$tmux kill-session -a
: Kills all the sessions apart from the active one.
$tmux kill-session
: Kills all the sessions.
tmux kill-server
: Kills the tmux server.
tmux-resurrect
and tmux-continuum
: Tmux plugins to persist sessions across restarts.
Prefix + s
(or tmux ls
followed by tmux attach -t SESSION
).Prefix + d
or tmux detach
tmux kill-session -t MY_SESSION
Prefix + c
or tmux new-window -n MY_WINDOW
Ctrl + d
or Prefix + x
Prefix + n
(next), Prefix + p
(previous) and Prefix + N
(where N
is the window index number, zero-based).tmux kill-window -t MY_WINDOW
Prefix + q
(or tmux list-panes
).Prefix + q
followed by the pane number you want to jump into.Prefix + o
Prefix + ;
tmuxinator is a tool that allows you to easily manage tmux sessions by using .yaml
files to describe the layout of a tmux session, and open up that session with a single command.
tmuxinator new [project]
: Create a new project file with given name and open it in your editor.tmuxinator list
: List all tmuxinator projects.tmuxinator copy [project] [copy_of_project]
: Copy an existing project to a new project and open it in your editor.tmuxinator delete [project_a] [project_b] ...
: Deletes given project(s).tmuxinator start -p your_tmuxinator_config.yml
: Start tmuxinator using custom configuration file (as opposed to it picking it up from default location like ˜/.config/tmuxinator\
).tmuxinator start console -n "console_jay" extra_param="any_string"
: Start a session console
, assign it project name “console_jay
” and extra arbitrary parameter “extra_param
” to pass value “any_string
”.tmuxinator stop [project]
: Stop a tmux session using a project’s tmuxinator config.ps
, pstree
, top
systemd
man systemd.unit
man systemd.service
man systemd.directives
hostnamectl status
hostnamectl hostname
: Query hostname.hostnamectl hostname <name>
: Change hostname.systemctl status
systemctl status bitcoind
systemctl start bitcoind
systemctl restart bitcoind
systemctl stop mempool fulcrum
systemctl enable --now bitcoind
systemctl disable bitcoind
systemctl list-unit-files --type service -all
: List all the services on your system and their status.systemctl --type=service --state=running
(where --state
can be any of running
, dead
, exited
, failed
or inactive
).systemctl daemon-reload
: Reload systemd files. If you change a service file in /etc/systemd/system/, they will be reloaded.systemctl cat bitcoind
: Show the service definition.systemctl show bitcoind
: Show the service parameters.journalctl -k
journalctl -b
journalctl --no-pager -b -p err
journalctl --list-boots
journalctl --utc
journalctl --since "2015-01-10" --until "2015-01-11 03:00"
journalctl --since yesterday
journalctl --since 09:00 --until "1 hour ago"
journalctl -p err -b
where option ‘-p’ can be any of:
0: emerg
1: alert
2: crit
3: err
4: warning
5: notice
6: info
7: debug
journalctl --no-full
journalctl --no-pager
journalctl -b -u bitcoind -o json
journalctl -b -u bitcoind -o json
journalctl -u bitcoind.service
journalctl -u bitcoind.service -u clightning.service --since today
What is the difference between IPtables and UFW Linux firewalls?
UFW is built upon IPtables, IPtables a very flexible tool but it’s more complex as compared to UFW. Also IPtables requires a deeper understanding of TCP/IP, which might not be the case with every Linux user, so UFW is the solution. UFW allows the user to configure firewall rules easily using IPtables under the hood. Hence, for an average Linux user UFW is the best way to get started with setting up different firewall rules.
We discuss and use UFW in our scope.
In NixOS the following commands are replaced by parameters in the configuration file (networking.firewall.allowTCPPorts
.etc…) you should not run them manually.
$ sudo ufw allow 9999
: Open port 9999.$ sudo ufw enable
: In case ufw
is not running (check with sudo ufw
status).$ netstat -ano
: See what ports are open and what processes uses to them.findmnt
: Lists all mounted filesytems or search for a filesystem. It is able to search in /etc/fstab, /etc/fstab.d, /etc/mtab or /proc/self/mountinfo. If device or mountpoint is not given, all filesystems are shown.
/
/mnt
/var
/etc
/tmp
/lib
time
: The simplest, shell built-in, tool to measure a command execution time.$ free -m
command to check your Linux memory usage.lshw
: Small tool to extract detailed information on the hardware configuration of the machine. It can report on memory, firmware version, mainboard configuration, CPU version and speed, cache configuration, bus speed, etc. in various output format. e.x. $ lshw -short
, $ lshw -json
…iperf3 -s
iperf3 -s -p port
iperf3 -c server
iperf3 -c server -P streams
iperf3 -c server -R
lscpu
: Part of util-linux
, command to display information about the CPU architecture.lsmem
: Part of util-linux
, command to list the ranges of available memory with their online status.memtester
: Effective userspace tester for stress-testing the memory subsystem. It is very effective at finding intermittent and non-deterministic faults.memusage
: Profile memory usage of a program.dmesg
: Shows Kernel Messages.dmesg -n 1
: Temporarily suppress all kernel logging to the console.$ inxi -Fxzb --usb
tree
: Outputs a depth-indented listing of files making it easy to visualize the organization of files and directories within a given path. With no arguments the tree lists the files in the current directory. When directory arguments are given, the tree lists all the files or directories found in the given directories each in turn.btop: Similar tool to glances
and htop
above.
Tailscale: Quick tutorial Rapidly deploy a WireGuard-based VPN, a “Zero-config VPN”: Automatically assigns each machine on your network a unique 100.x.y.z IP address, so that you can establish stable connections between them no matter where they are in the world, even when they switch networks, and even behind a firewall. Tailscal nodes use DERP (Designated Encrypted Relay for Packets) to proxy encrypted WireGuard packets through the Tailscale cloud servers when a direct path cannot be found or opened. It uses curve25519 keys as addresses.
$ ngrok config add-authtoken TOKEN
$ ngrok http 8000
$ systemctl status bitcoind
bitcoin-cli
is the most straightforward way to execute bitcoin RPC commands (full list of RPC commands), here are some of the most commonly used:
$ bitcoin-cli -getinfo
$ bitcoin-cli getblockchaininfo
jq
to get pretty JSON output formatting or extract specific data, for instance:
$ bitcoin-cli getblockchaininfo | jq '.'
$ bitcoin-cli getblockchaininfo | jq '.verificationprogress'
$ bitcoin-cli help
$ bitcoin-cli help getblockchaininfo
$ bitcoin-cli getblockchaininfo
$ bitcoin-cli getpeerinfo
$ bitcoin-cli getnetworkinfo
$ bitcoin-cli getmininginfo
$ bitcoin-cli getblockcount
$ bitcoin-cli getbestblockhash
$ bitcoin-cli getblock <hash>
$ bitcoin-cli getblockhash <index>
$ bitcoin-cli getwalletinfo
$ bitcoin-cli createwallet
$ bitcoin-cli listreceivedbyaddress 0 true
: List of accounts on the system.$ bitcoin-cli setaccount 1GBykdD628RbYPr3MUhANiWchoCcE52eW2 myfirstaccount
: To associate an existing address (here : 1GBykdD628RbYPr3MUhANiWchoCcE52eW2) to an account name.$ bitcoin-cli sendfrom myfirstaccount 1AYJyqQHCixxxxxxffevxxxxQosCWqn1bT 0.15
: Send bitcoins (here : 0.15) to an address (here : 1AYJyqQHCixxxxxxffevxxxxQosCWqn1bT)$ bitcoin-cli getrawmempool
$ bitcoin-cli getrawtransaction <txid>
$ bitcoin-cli decoderawtransaction <rawtx>
The bitcoin JSON-RPC API allows to interact with bitcoin deamon in a varierty of ways: cURL, JavaScript, Python …
First you need to make sure Bitcoin Core is setup to accept Remote Procedure Calls (RPC):
For instance you could run getnetworkinfo
using the following cURL:
curl -u public:2S8PWBZ71wMXdrsAxL21 -d '{"jsonrpc": "1.0", "id": "curltest", "method": "getnetworkinfo", "params": [] }' -H 'content-type: text/plain;' http://mybonk-jay:8332/
Most noticably you need to use the option -u
(or --user
) to pass valid crenentials (here username public
and password 2S8PWBZ71wMXdrsAxL21
) allowing you to connect else you will get an 401 Unauthorized
error. Username is either public
or priviledged
, their password in /etc/nix-bitcoin-secrets/bitcoin-rpcpassword-{public|priviledged}
.
Also make sure that the method you call (getnetworkinfo
, getpeerinfo
, listwallets
…) is indeed in the RPC whitelist else you will get a
You can use curl with the -v
(verbose) parameter to see the headers sent: The text after the Basic
keyword is the base64 encoded text string of the username:password
combination that was passed with the -u
parameter.
Authorization: Basic cHVibGljOjJTOFBXQlo3MXdNWGRyc0F4TDIx
To manually generate the base64 encoded credentials on Linux, you can simply call:
$ echo -n "username:password" | base64 -w0
cHVibGljOjJTOFBXQlo3MXdNWGRyc0F4TDIx
To test this end to end, you can remove -u username:password
and substitute with -H Authorization: Basic cHVibGljOjJTOFBXQlo3MXdNWGRyc0F4TDIx
and it will still authenticate just fine:
$ curl -v -d '{"rpc": "1.0", "id": "curltest", "method": "getnetworkinfo", "params": [] }' -H 'content-type: text/plain;' -H 'Authorization: Basic cHVibGljOjJTOFBXQlo3MXdNWGRyc0F4TDIx' http://mybonk-jay:8332/
In conculsion, you could even run these RPC commands without using cURL: You would just need to base64 encode the username:password
combination and set the HTTP Authorization
header with the type as Basic
along with the base64 encoded string.
This is a nice cheatsheet for https://github.com/grubles/cln-cheatsheet
$ systemctl status clightning
lightning-cli
lightning-cli
is simply a wrapper over core-lightning JSON RPC to interact with the lightning deamon, and print the result.
$ lightning-cli --version
$ lightning-cli help
$ lightning-cli --version
$ lightning-cli help
$ lightning-cli getchaininfo
$ lightning-cli getinfo
Something fun to do is to stream stats from one node (the “source”) to another (the “destination”):
On the “source” machine run the following command to send 100 times between 2000 and 65000 sats using keysend to node with pubkey 0278e764e98cf94ef1f33684bac0f7cc85b3d445465cc4c0171d07107079aaf4cc
(replace by the pubkey of your destination node):
$ for i in {1..100}; do lightning-cli keysend 0278e764e98cf94ef1f33684bac0f7cc85b3d445465cc4c0171d07107079aaf4cc `shuf -i 2000000-65000000 -n 1`; done
Now on the “destination” machine run the following command to observe lightning channels grow as the payments are processed:
$ watch "lightning-cli listfunds | jq '.channels[] | .our_amount_msat'"
Another example is sending funds onchain using lightning-cli, which is also possible.
bech32
address format by default):
$ lightning-cli newaddr
{
"bech32": "tb1qmws65ajdzfk3etqzumfk9ujumjhj8tgvk7rwys"
}
$ lightning-cli withdraw tb1qmws65ajdzfk3etqzumfk9ujumjhj8tgvk7rwys 10000000
The command will take a couple of seconds to complete and will return the txid
(transaction id) along with other data. The command withdraw
has a few optional parameters, have a look at its documentation.
Similarly to the bitcoin JSON-RPC, the clightning JSON-RPC API allows to interact with clightning deamon in a varierty of ways: cURL, JavaScript, Python …
Corelightning documentation is very well done with clear examples.
If you are actualy running NixOS operating system (as opposed to just Nix which is the package manager), you can lookup the currently active NixOS configuration. For example:
$ nixos-version
23.05.20230915.360a7d3 (Stoat)
Of course you also use the traditional Linux commands to know more:
cat /etc/os-release
BUG_REPORT_URL="https://github.com/NixOS/nixpkgs/issues"
BUILD_ID="23.05.20230915.360a7d3"
DOCUMENTATION_URL="https://nixos.org/learn.html"
HOME_URL="https://nixos.org/"
ID=nixos
LOGO="nix-snowflake"
NAME=NixOS
PRETTY_NAME="NixOS 23.05 (Stoat)"
SUPPORT_END="2023-12-31"
SUPPORT_URL="https://nixos.org/community.html"
VERSION="23.05 (Stoat)"
VERSION_CODENAME=stoat
VERSION_ID="23.05"
and:
$ hostnamectl
Static hostname: mybonk-jay
Icon name: computer-vm
Chassis: vm 🖴
Machine ID: b93a6b5ab0384df583b2f691e312314c
Boot ID: 82b819d538bd4a1a93ec046fc81111e5
Virtualization: kvm
Operating System: NixOS 23.05 (Stoat)
OS Support End: Sun 2023-12-31
OS Support Expired: 10month 5d
Kernel: Linux 6.1.52
Architecture: x86-64
Hardware Vendor:
Hardware Model: vServer
Firmware Version: 20171111
Firmware Date: Sat 2017-11-11
nix --version
: Get running nix version.nix-info -m
nix-shell
: Start an interactive shell based on a Nix expression. This is distinct from nix shell
(great explanation of the difference between ‘nix-shell’ and ‘nix shell’).nix-build
: Build a Nix expression. This is distinct from nix build
.nix-channel
nix-collect-garbage
nix-copy-closure
nix-deamon
nix-env
nix-hash
nix-instantiate
(same as nix-instantiate default.nix
)
nix-instantiate --eval
: Very easy way to evaluate a nix file.nix-instantiate --eval --strict
: --strict
tries to evaluation the entire result (otherwise may return <CODE>
blocks).nix-instantiate --eval --json --strict
: Always use --strict
with --json
(otherwise <CODE>
blocks may result in json not being able to parse).nix eval -f default.nix
. Note that nix eval behaves as --strict
(tries to evaluation the entire result).nix-prefetch-url
nix-store
nix repl
to interactively explore the Nix language as well as configurations, options and packages.
$ nix repl
Welcome to Nix 2.15.0. Type :? for help.
nix-repl>
:?
to get help on the commandsUse :q
to quit nix-repl.
:doc
, for instance:
nix-repl> :doc dirOf
:log
, for instance:
nix-repl> builtins.readFile drv
"Hello world"
nix-repl> :log drv
Hello world
$ nix repl --file '<nixpkgs/nixos>' -I nixos-config=./configuration.nix
$ nix repl>:l <nixpkgs/nixos>
$ nix repl>:lf /etc/nixos
$ nix repl>nixosConfigurations.<hostname>
keep-derivations
(default: true
) and keep-outputs
(default: false
) in the Nix configuration file.nix-collect-garbage --delete-old
: Quick and easy way to clean up your system, deletes all old generations of all profiles in /nix/var/nix/profiles
. See the other options below for a more “surgical” way to garbage collect.nix-env --delete-generations old
: Delete all old (non-current) generations of your current profile.nix-env --delete-generations 10 11 23
: Delete a specific list of generationsnix-env --delete-generations 14d
: Delete all generations older than 14 days.nix-store --gc --print-dead
: Display what files would be deleted.nix-store --gc --print-live
: Display what files would not be deleted.nix-env
with an argument --delete-generations
) - you can run the garbage collector as follows: nix-store --gc
lib.debug.traceSeq <arg1> <arg2>
: Print a fully evaluated value.You can search for packages using search.nixos.org:
But you can also search for packages straight from the command line, there are 2 methods to do this:
Just use nix-shell
’s autocomplete feature: Press the < tab > key as you enter the name, you’ll see all Nix package names that begin with the text you entered (takes a second or two to complete):
$ nix-shell -p asciiq
asciiquarium
Note: Nix shell autocomplete applies only on the Nix package names (not its description nor any other metadata).
Use the nix
terminal app with search
:
$ nix search nixpkgs asciiquarium
* legacyPackages.x86_64-linux.asciiquarium (1.1)
Enjoy the mysteries of the sea from the safety of your own terminal!
* legacyPackages.x86_64-linux.asciiquarium-transparent (2023-02-19)
An aquarium/sea animation in ASCII art (with option of transparent background)
Notes:
nix search
does a full-text search.nix search nixpkgs asciiquarium
else it is nix search asciiquarium
.--json
flag to get more information in the output (and pipe through jq
to get it in a nice readable format):
$ nix search nixpkgs --json asciiquarium | jq
evaluating 'legacyPackages.x86_64-linux'
evaluating 'legacyPackages.x86_64-linux'{
"legacyPackages.x86_64-linux.asciiquarium": {
"description": "Enjoy the mysteries of the sea from the safety of your own terminal!",
"pname": "asciiquarium",
"version": "1.1"
},
"legacyPackages.x86_64-linux.asciiquarium-transparent": {
"description": "An aquarium/sea animation in ASCII art (with option of transparent background)",
"pname": "asciiquarium-transparent-unstable",
"version": "2023-02-19"
}
}
This is a great feature of NixOS. For example I can open a shell with the asciiquarium
package available in it:
$ nix-shell -p asciiquarium
Now get this: that command didn’t actually install anything. When I leave the shell (with exit
or ctrl+d
), asciiquarium
is no longer there, and it doesn’t pollute my user environment.
Alternatively, if you have Flakes enabled you can us the following command (more information here):
$ nix shell nixpkgs#asciiquarium
You can play around with plenty of nifty little programs kids love:
$ nix-shell -p asciiquarium cmatrix fortune cowsay sl figlet toilet oneko
.etc…
asciiquarium
: Enjoy the mysteries of the sea from the safety of your own terminal.cmatrix
: Shows a scrolling ‘Matrix’ like screen on the terminal.fortune
: Simple program that displays random poignant, inspirational, silly or snide phrases from a database of quotations.cowsay
: Simple little ASCII art tool that prints a picture of a cow with a speech bubble message; you can enter the following command to “pipe” the output of the aforementioned fortune
command to create your own wise cow. Try out $ fortune | cowsay
sl
: Animated ASCII steam locomotive with a few nice hidden optionsfiglet
: ASCII banner creatorpipes
: Animated pipes terminal screensaver.toilet
: More ASCII fonts. For some reason, they named it toilet.oneko
: Adds an animated cat to your terminal who will follow your mouse (get it?)bastet
: Tetris on the command-line.ninvaders
: Space Invaders on a command-line.moon-buggy
: Jump, fire, on a command-line.Many other little games are bundled in a package. The package bsdgame
for instance has cool command-line games.
nix-community/awesome-nix: “A curated list of the best resources in the Nix community.”.
If you are using an SSD it may be useful to enable TRIM support as well as set filesystem flags to improve the SSD performance: ` fileSystems.”/”.options = [ “noatime” “nodiratime” “discard” ]; `