Guides on dd

Newsboat

Fri, 19 Aug 2022 14:31:59 -0300

Allthough I’ve riced newsboat, this will be a guide focusing on the important aspect of the program. Newsboat allows the user to read RSS/Atom feeds, which are usually generated by the website itself or sometimes by a frontend or third party app. The main reason to use this program is the fact that you won’t need an account in sites such as Youtube, Reddit, or Twitter anymore. Also, that you’ll have sort of a centralized way to consume (which includes reading text, listening to podcasts or watching videos) pretty much any site on the internet by using only a terminal program, which is by far faster and more desirable. This guide includes:

Brief explanation on how it works (program is very intuitive to use)
Setup of other programs
Running newsboat in the background so it notifies the user when new article appears (optional)
Multiple macro configurations that you might find very useful (optional but HIGHLY recommended)

Installation

The software is at almost any repository. In case it is not on your distro, you can always build it from source.

For Arch-based systems:

pacman -S newsboat mpv

I use firefox for opening up links (unless is the article has only text) and mpv to reproduce videos and audios. You can use any other browser and media player obviously. Or, you can use something like w3m to read text, but remember to change it as the browser when setting up the program.

Also, if you want to download videos/audios I recommend installing yt-dlp from its github’s repository. The installation is straightforward and the software is easier to update by using yt-dlp -U once needed. This program is a fork of the discontinued youtube-dl which may still appear in some distro’s repositories.

Configuration

Newsboat

Newsboat won’t run unless the file urls has something inside. Both config and urls files are at either $HOME/.newsboat or $XDG_CONFIG_HOME/newsboat. Put something inside the urls file. I recommend using the following frontends for getting the feeds:

Twitter -> nitter
Youtube -> Invidious
Reddit -> teddit

Say you want to add the youtube channel HydeWars to your feed. It will look like this:

https://vid.puffyan.us/feed/channel/UCfUaZ8Ra7m7BqUEACv2jySw

So basically, you need to get the channel’s ID which is UCfUaZ8Ra7m7BqUEACv2jySw and add it at the end of the url, where vid.puffyan.us is an instance of Invidious. How you find the ID of a YT channel is a matter of having at least a 2 digits IQ. If you don’t want to use an Invidious instance, you can go to any Youtube channel and view the souce code, filter keyword rss and see how the URL looks.

You can also tag URLs and then press t by adding the tag at the end of the string after a blank space:

https://vid.puffyan.us/feed/channel/UCfUaZ8Ra7m7BqUEACv2jySw MDE

Now open config file and add:

# GENERAL #
reload-time 30 
auto-reload yes
browser "setsid -f $BROWSER --new-tab %u > /dev/null 2>&1"
cleanup-on-quit yes
history-limit 2000
show-keymap-hint no
goto-next-feed no
error-log ".config/newsboat/error.log"
prepopulate-query-feeds yes
suppress-first-reload yes

# NOTIFICATIONS #
notify-always no 
notify-program "/usr/bin/dunstify"
notify-format "Newsboat: %d new articles"

Line 4 sets up the browser and forks it, while opening the url in a new tab. In my case, I have an env var set to firefox. Change $BROWSER to your browser’s name or software for reading text, if you need to. Then, for notifications I use dunstify but you can use whatever you like. Rest is self explanatory, but keep in mind that if you are going to setup notifications you should keep reload-time and auto-reload as they are.

Optionally, you can set up vim-like bindings:

# Vim keybindings
unbind-key j
unbind-key k
unbind-key ENTER
unbind-key o 

bind-key o open 
bind-key k up
bind-key j down

Mpv

Open $XDG_CONFIG_HOME/mpv/mpv.conf and add:

# Cache
cache=yes
--stream-buffer-size=8MiB

# Quality stream
ytdl-format=bestvideo[height<=?720]+bestaudio/best

# Yt-dlp hook
script-opts-append=ytdl_hook-ytdl_path=yt-dlp

This sets up a yt-dlp hook that will make the streaming faster. Also, if you want higher/lesser quality, change the height value.

Yt-dlp

I recommend you to set up a download folder. Open up $XDG_CONFIG_HOME/yt-dlp/config and add:

-o '/path/to/folder/%(title)s.%(ext)s'

This will save the video/audio to a folder using metadata.

Running newsboat through a script for notifications

Instead of running newsboat directly, I use a simple script so it is always on the background. You can also achieve this with cronjobs.

#!/bin/sh

while true; do
    kill $(pidof newsboat)
    rm $XDG_CONFIG_HOME/newsboat/queue
    $TERMINAL -e newsboat
    if [[ $? == 0 ]] ; then
        exec newsboat && break
    else
        break
    fi
done

chmod +x the script and remember to use it instead of directly executing newsboat. In my case, I use an i3’s keybinding for quick access, and also for executing the script only one time as soon as the window manager initializes.

Macros

A macro is used for executing a sequence of commands by pressing a key or a combination of keys. In our case, for using the browser setting as not really a browser, but anything we like. For example, as a media player to reproduce a YT video. To execute a macro press , + key.

Here is a list of some macros I’ve came up with that are very useful (add them to newsboat’s config file):

Queue videos, clear playlist and reproduce playlist

macro a set browser "echo %u >> ~/.config/newsboat/queue" ; open-in-browser ; set browser "$BROWSER %u"
macro c set browser "rm $HOME/.config/newsboat/queue > /dev/null 2>&1" ; open-in-browser ; set browser "$BROWSER %u"
macro p set browser "kill $(pidof mpv) ; setsid -f mpv --playlist=$HOME/.config/newsboat/queue > /dev/null 2>&1" ; open-in-browser ; set browser "$BROWSER %u"

The idea of these 3 macros is creating, playing or deleting a playlist. If you take a look at the script in the previous section, this file named queue gets deleted when the script executes. For adding videos or even audios to said file you need to focus the article on a feed.

Play queued videos fullscreen second monitor

macro P set browser "kill $(pidof mpv) ; setsid -f mpv --x11-name=newsboatfs --fullscreen=yes --playlist=$HOME/.config/newsboat/queue > /dev/null 2>&1" ; open-in-browser ; set browser "$BROWSER %u"

Play video

macro v set browser "kill $(pidof mpv) ; setsid -f mpv %u > /dev/null 2>&1" ; open-in-browser-and-mark-read ; set browser "$BROWSER %u"

Play video floating mode

macro i set browser "kill $(pidof mpv) ; setsid -f mpv --x11-name=mpvfloat %u > /dev/null 2>&1" ; open-in-browser-and-mark-read ; set browser "$BROWSER %u"

Play audio only

macro A set browser "kill $(pidof mpv) ; setsid -f mpv %u --no-video > /dev/null 2>&1" ; open-in-browser-and-mark-read ; set browser "$BROWSER %u"

Play fullscreen, i3 sends it to second monitor (useful for playing all the videos from a channel)

macro f set browser "kill $(pidof mpv) ; setsid -f mpv --x11-name=newsboatfs --fullscreen=yes %u > /dev/null 2>&1" ; open-in-browser ; set browser "$BROWSER %u"

Download video

macro y set browser "yt-dlp %u" ; open-in-browser ; set browser "$BROWSER %u"

Open in default browser

macro o open-in-browser

NOTE: Keep in mind that for sending the videos/articles to another monitor you need to rename the X instance to whatever you like so you can then manipulate it with your window manager. Using i3 would look like this:

for_window [instance="newsboatfs"] move container to workspace $ws10
for_window [instance="mpvfloat"] floating enable, resize set 960 540, move container position center

Where $ws10 outputs to HDMI-1:

workspace "10" output HDMI-1

Use xrandr to know display names.

Firefox

Mon, 16 May 2022 22:05:11 -0300

A guide and explanation for making Firefox more secure/private using arkenfox user.js and some essential addons. This is a compilation from various sources that are linked at the bottom of this article, and from my useless and extensive attempt for having a useful yet ‘privacy-oriented’ and ‘secure browser’, things that are mutually exclusive. Still, this guide will leave the user with a better than nothing tool to navigate the net.

A little test before

You should check your browser against fingerprinting just so you can compare after. For that use this website: deviceinfo.me. This is all the data that first-party and third-party sites get from you, but we will minimize it. Keep in mind that some information won’t be concealed, such as your IP or location. Please do the test again after you finish.

arkenfox user.js

LINK

This tool is just a user config template that interacts with the inner functions of Firefox. It is highly recommended that you read the wiki so you can customize it. Otherwise, with just downloading the file and making the browser use it would be more than enough in most cases. So for that:

firefox -no-remote -CreateProfile <userprofile>

That will create a user directory under $HOME/.mozilla/firefox/ that contains the string <userprofile> at the end of it. Now delete its content, download arkenfox user.js and activate the profile:

cd $HOME/.mozilla/firefox/<userprofile>/ && rm times.json
wget https://raw.githubusercontent.com/arkenfox/user.js/master/user.js
firefox -P <userprofile>

Note: firefox -P <userprofile> where <userprofile> is just the string you used to create the profile (not the random numbers from the directory)

Check /usr/lib/firefox/ for these plugins (some may not be included) and delete them:

firefox@getpocket.com.xpi
followonsearch@mozilla.com.xpi
activity-stream@mozilla.org.xpi
screenshots@mozilla.org.xpi
onboarding@mozilla.org.xpi
formautofill@mozilla.org.xpi
webcompat@mozilla.org.xpi

Those are the basics, as I said read the extense wiki for customizing the template.

Note: notice that the content of the explorer have borders. That is a letterboxing option that strengthens against fingerprinting. If it bothers you, edit your user.js and search for user_pref("privacy.resistFingerprinting.letterboxing", true);. Then replace true with false.

Now start firefox we are going to install some addons.

uMatrix

LINK

The superior blocker. If configured properly, it will restrict any malicious site you may misstakenly enter to and block any pop up window or annoying ad, guaranteed. Other extensions or even the built-in anti ad options of for example, Brave Browser, are useless and do not work properly.

This extension gives you a per site list that shows first and third party domains that you are establishing a connection to. If you click on the extension icon and look at the grid, you will notice 8 elements. These are simply the reason why this addon is superior to others. It will block ANYTHING because it doesn’t block per domain. In other words and as an example, if you deny script, it will block JavaScript in every site you visit. Inferior extensions have a gigantic database of domains to work with, so if a domain is missing it is impossible for it to block its elements.

First, go to uMatrix’s configuration panel and open the Settings tab. Copy this:

I recommend you use Domain as an option to Default scope level so you can create more flexible rules such as the one from the example that comes later in the guide. Cookies are trapped locally by uMatrix. This allows you to inspect the contents of it and blocks the sender from getting it back. Turn on the option and set a timer for deleting non-blocked cookies if you want.

Moving on, lets generate some rules. Go to uMatrix’s panel and then to My Rules. Observing, you can see two sections: left is for permanent rules and right is for temporary rules. For editing a rule in, type in the right section, then save it and click commit. Rules consist of 4 parts (* is a wildcard, which means any):

* * * allow/deny
| | |
| | |______ Element
| |________ Domain
|__________ Scope

So for a start, consider adding these strong rules:

* * * block
* * css allow
* * image allow

The first line blocks ANYTHING on any scope and domain. Then the second and third overrides first and allows css and image on ANY scope and domain. Pretty easy. This is a good start for then tweaking and adding more rules.

Say you want to log in to a site you frequent. This site will need cookies allowed, and maybe needs a script to run a captcha from a third-party domain like google. Such rule would look like this:

* ineedcookies.com cookie allow 
ineedcookies.com googlecaptcha.com script allow

This will allow the google’s domain only in the site requesting for a login, which is desirable. This is pretty much it, if you are not looking for a strong blocking ruleset, you can use uBlock Origin which is from the same creator, or search the wiki for a more suitable example.

Decentraleyes

LINK

This one is a content blocker that will deny, in the majority of cases, third party domains from trying to deliver something you don’t need. You could say, ‘but isn’t that already done by uMatrix?’, and you are honestly right. The thing is that uMatrix breaks everything it touches. While adding Decentraleyes on top of it, you could still mantain some aspects of the sites you are visiting. Decentraleyes stores content locally so you can still make use of it without the sender getting a response. On the other hand, uMatrix will prevent Decentraleyes from doing so if it is hardblocking content. To avoid that from happening, you need to create some extra rules in uMatrix that allow traffic to some convenient domains.

These are the rules:

* ajax.aspnetcdn.com * allow
* ajax.googleapis.com * allow
* ajax.microsoft.com * allow
* cdn.jsdelivr.net * allow
* cdnjs.cloudflare.com * allow
* code.jquery.com * allow
* lib.sinaapp.com * allow
* libs.baidu.com * allow
* upcdn.b0.upaiyun * allow
* yandex.st * allow
* yastatic.net * allow

More rules could and should be added as long as you keep using the extensions.

Privacy Redirect

LINK

This one is a redirector for the most famous and used sites such as Twitter, Reddit or Youtube. Simply click on the icon and turn on/off which service you want to redirect to its respective frontend. Frontends are very useful at times when you can’t view content that is age restricted or simply because you are not logged in, not to mention that you also skip the annoying pop up windows from shitsites like Twitter. This shouldn’t be a problem since you are running uMatrix now, but it is good to know. Also, using a frontend like Invidious for Youtube, allows for navigation with no ads, no tracking (doesn’t log your IP) and without JS enabled.

Here is a quote from Nitter’s about section (logic applies to the other frontends):

It’s impossible to use Twitter without JavaScript enabled. For privacy-minded folks, preventing JavaScript analytics and IP-based tracking is important, but apart from using a VPN and uBlock/uMatrix, it’s impossible. Despite being behind a VPN and using heavy-duty adblockers, you can get accurately tracked with your browser’s fingerprint, no JavaScript required. This all became particularly important after Twitter removed the ability for users to control whether their data gets sent to advertisers. Using an instance of Nitter (hosted on a VPS for example), you can browse Twitter without JavaScript while retaining your privacy. In addition to respecting your privacy, Nitter is on average around 15 times lighter than Twitter, and in most cases serves pages faster (eg. timelines load 2-4x faster). In the future a simple account system will be added that lets you follow Twitter users, allowing you to have a clean chronological timeline without needing a Twitter account.

This is the list of sites that the extension allows to redirect:

Twitter → Nitter
Youtube → Invidious
Instagram → Bibliogram
Reddit → Libreddit or old version
Google Translate → Simply Translate
Wikipedia → Wikiless
Google Maps → OpenStreetMaps
Search Engine → custom

I recommend you go to the general options, where you can set the instance of the frontend you want to use.

We are done. Remember to run the test again and compare to see the results.

Stylus: custom/community generated css with one click
ff2mpv: forward links to mpv (useful for when you break js/xhr/frame on sites that have videos)
Vimium-FF: vimlike bindings

Prosody

Fri, 13 May 2022 18:23:51 -0300

This guide is for installing Prosody, an XMPP server that is decentralized, fast, simple and FOSS. The version we will be using is 0.11.12 and in the end the user will have a private and only c2s server. These options are of course changeable after or during the installation.

Prerequisites

GNU/Linux system
VPS (recommended) or a home server
Domain name
Basic terminal knowledge

Installation

We install the main packages plus some extras for TLS encryption, A/V streaming, and file transfering. If you don’t care about these things you can skip them.

On Debian/Ubuntu:

apt install prosody prosody-modules python3-certbot-nginx coturn mercurial

prosody is the main package
prosody-modules are some extra packages for functionability
python3-certbot-nginx is for TLS encryption
coturn a STUN/TURN server that allows A/V streaming for users behind NAT
mercurial for installing community modules for the STUN/TURN server

Configuration

The server’s CFG file is at /etc/prosody/prosody.cfg.lua.

Admin, users and the domain name

...
admins = { "admin1@domain.org", "admin2@domain.org" }
...
VirtualHost = "domain.org"
...

Now from the terminal add some users:

prosodyctl adduser user@domain.org

The program will prompt for a password. To delete a user use the command deluser, and for changing passwords use passwd, both with the JID as an option.

Modules enabled/disabled, user registration

Search for the line modules_enabled and add the modules http_files (file transfer), turn_external (STUN/TURN server) and uncomment csi_simple and disco if they are commented. Under modules_disabled only leave s2s uncommented. Finally, check if in the following lines allow_registration is set to false, which is self explanatory.

File transfering

We will be configuring two components in the CFG file. You should add them after the VirtualHost section.

Component "upload.domain.org" "http_upload"

Right after we add http_upload_file_size_limit = 20971520 and http_upload_expire_after = 60 * 60 * 24 * 7, for limiting the file size and setting its expiration.

Now, in the global section (before VirtualHost) add:

-- HTTP/HTTPS ports
http_ports = { 5280 }
http_interfaces = { "*", "::" }

https_ports = { 5281 }
https_interfaces = { "*", "::" }

If it is your case, remember to configure your firewall accordingly.

After VirtualHost we add:

disco_items = {
	{ "upload.domain.org", "File Sharing Service" },
}

In the components section:

Component "proxy.domain.org" "proxy65"
proxy65_address = "domain.org"

There is no need to add proxy65 to the modules_enabled list. This component lets users behind NAT transfer files.

Coturn: The STUN/TURN server

Check if coturn is running:

systemctl status coturn

If not start it:

systemctl enable --now coturn

Next thing to do is downloading and setting the correct modules from the community repository using mercurial.

hg clone https://hg.prosody.im/prosody-modules/ prosody-modules

Now you can either copy (not recommended) the modules mod_turn_external.lua and mod_external_services.lua to /usr/lib/prosody/modules or create another folder for the community plugins that will be installed and create symlinks for them. For the second option, add the created folder to the plugins path in prosody.cfg.lua:

plugins_path { "usr/lib/prosody/modules", "enabled/plugins/folder" }

Create the symlinks from the community downloaded folder to your plugins enabled folder (it depends on where you downloaded those modules):

ln -s /downloadedfolder/mod_turn_external/mod_turn_external.lua /enabled/folder
ln -s /downloadedfolder/mod_external_services/mod_external_services.lua /enabled/folder

We edit the coturn cfg file that is located in /etc/turnserver.conf:

realm=turn.domain.org
static-auth-secret=yoursecretpassword

Finally uncomment use-auth-secret

We go back to our prosody.cfg.lua file. In the global section add:

turn_external_host = "turn.domain.org"
turn_external_secret = "yoursecretpassword"

VERY IMPORTANT: Certificates

We need to generate certificates for the domain and every subdomain we are using for our components. Also, we need to check for some configuration options that could be missing or commented.

First we generate:

certbot -d domain.org --nginx
certbot -d upload.domain.org --nginx
certbot -d proxy.domain.org --nginx
certbot -d turn.domain.org --nginx

The bot will give you some output in the terminal and prompt you for two options: select the second one every time.

Now, we need to import/install the certs to prosody:

prosodyctl --root cert import /etc/letsencrypt/live/

The TLS encryption for the file transfering module needs to be explicitly configured, and for that we edit prosody.cfg.lua and add to global:

https_ssl = {
	certificate = "/etc/prosody/certs/upload.domain.org.crt";
	key = "/etc/prosody/certs/upload.domain.org.key";
}

Pay attention to the extension names and double check that you got the right path and files for each line.

Inside the same file, check the following line and set it to true:

c2s_require_encryption = true

We are done with our file transfering configuration.

For the STUN/TURN server we also need to modify its CFG file /etc/turnserver.conf to set a path for our certs:

cert=/etc/letsencrypt/live/turn.domain.org/fullchain.pem
pkey=/etc/letsencrypt/live/turn.domain.org/privkey.pem

Done. You can check for errors using prosodyctl check. As a final note, I should add that if you are using a VPS you probably have a firewall working. There are some ports that need to be forwarded: 5280, 5281, 5222, 5322, 5000, 3478. If you are not using a firewall I recommend you using ufw and start from there.

Also, that this configuration is very personal. You can add more components (for example multichat groups). For that you should RTFM, which is always ideal.

Prosody Docs

Guides on dd

Newsboat

Installation

Configuration

Newsboat

Mpv

Yt-dlp

Running newsboat through a script for notifications

Macros

Queue videos, clear playlist and reproduce playlist

Play queued videos fullscreen second monitor

Play video

Play video floating mode

Play audio only

Play fullscreen, i3 sends it to second monitor (useful for playing all the videos from a channel)

Download video

Open in default browser

Firefox

A little test before

arkenfox user.js

uMatrix

Decentraleyes

Privacy Redirect

Not privacy/security related addons

Links

Prosody

Prerequisites

Installation

Configuration

Admin, users and the domain name

Modules enabled/disabled, user registration

File transfering

Coturn: The STUN/TURN server

VERY IMPORTANT: Certificates