Java has the basic array type which is just any type suffixed with [] to make it an array. To copy an array like it you use the static method arraycopy from the System class.
class Main {
public static void main(String[] args) {
byte[] a = {1, 2, 3, 4, 5, 6, 7, 8, 9, 0};
byte[] b = new byte[10];
System.arraycopy(a, 0, b, 0, 10);
a[9] = 10; // Change something to make sure we are printing the original values
for (byte c: b) {
System.out.print(c);
System.out.print(" ");
}
System.out.println();
}
}
Output:
1 2 3 4 5 6 7 8 9 0
But keep in mind that such an array is not like a java primitive so it gets passed as is into functions without it being copied at any stage like a primitive type would.
class Main {
public static byte[] dosomething(byte[] a) {
return a;
}
public static void main(String[] args) {
byte[] a = {1, 2, 3, 4, 5, 6, 7, 8, 9, 0};
byte[] b = dosomething(a);
b[9] = 10;
for (byte c: b) {
System.out.print(c);
System.out.print(" ");
}
System.out.println();
}
}
Output
1 2 3 4 5 6 7 8 9 10
Java also has classes like ArrayList which is a dynamic array and to copy one you use the clone method.
class Main {
public static void main(String[] args) {
ArrayList l = new ArrayList();
l.add(1);
l.add(2);
l.add(3);
ArrayList l2 = (ArrayList)l.clone();
l.add(4);
for (Object c: l2.toArray()) {
System.out.println(c);
}
}
}
Output
1
2
3
JavaScript has as of a few years back something called the spread operator which you write as ... and it can be used on iterable objects to get all members.
a = [1,2,3]
// Array(3) [ 1, 2, 3 ]
b = [...a]
// Array(3) [ 1, 2, 3 ]
a[0]=0
// 0
a
// Array(3) [ 0, 2, 3 ]
b
// Array(3) [ 1, 2, 3 ]
Python has the basic list dynamic array type which uses the [] notation to construct it but you can call the constructor like list() too. To make a copy it you can use the [:] slice notation which does various forms of shallow copies of the list.
a = [1, 2, 3]
b = a[:]
a[0] = 0
print(b)
# [1, 2, 3]
a = b[::-1]
print(a)
# [3, 2, 1] Yes it got reversed
a = a[1:2]
print(a)
# [2]
a = b[::2]
print(a)
# [1, 3]
a = b[-2::]
print(a)
# [2, 3]
Up to 3 numbers are specified in the slice notiation of the form [start:end:skip] where start is the start position (inclusive) and end is the end position (exclusive) and skip is how many steps it will take between and negative is allowed in all the arguments where a negative skip will reverse. Negative starts will start reading at the specified index at the end of the list.
Python also has an array type called array but in all the decades of me programming in Python I have used it about once but this is how you use it together with the copy module with calls the __copy__ special method.
import array
import copy
a = array.array("i", [1, 2, 3])
print(a)
# array('i', [1, 2, 3])
b = copy.copy(a)
a[0] = 0
print(b)
# array('i', [1, 2, 3])
The copy module also supports deep copying which is very convenient for when you actually want to deep copy an object.
import array
import copy
a = [['A', 'B', 'C'], 1, 2, 3]
b = copy.deepcopy(a)
a[0].insert(0, 0)
print(a)
# [[0, 'A', 'B', 'C'], 1, 2, 3]
print(b)
# [['A', 'B', 'C'], 1, 2, 3]
C is a bit unusual compared to most modern languages as you operate pretty much in raw memory. This means that you can cast any type to any other type and this means also that you do not need to know the type of the thing you copy with memcpy but you do need to know the memory areas and the size of the array in memory.
#include <stdio.h>
#include <string.h>
void printArray(int* a, int len) {
for(int i = 0; i < len; i++) {
printf("Position %d = %d\n", i, a[i]);
}
}
int main() {
int a[] = {1, 2, 3};
int b[3];
memcpy(b, a, sizeof(a));
a[0] = 0;
printArray(b, 3);
printf("%d is the size of the array in memory\n", sizeof(a));
}
Output
Position 0 = 1
Position 1 = 2
Position 2 = 3
12 is the size of the array in memory
You might be confused why 12 is the memory size of the array when it has only 3 elements. The reason for that is that an integer in this version of C on this platform has a memory size of 4 bytes. The platform is ARM 64 bit here if you wonder. When it comes to stuff like casting and memory it has been common for some C programmers to create a struct type for example then cast it to a byte array to send over network and that is the reason why the struct module in Python is called that.
Want to copy an array in PHP? The bigger question is how to not copy an array in PHP. Assigning a variable to another variable will copy it almost completely.
<?php
$a = [1, 2, 3];
$b = $a;
$c = &$a;
$a[] = 4;
echo json_encode($a), "\n";
# [1,2,3,4]
echo json_encode($b), "\n";
# [1,2,3]
echo json_encode($c), "\n";
# [1,2,3,4]
If you thought it stopped at a shallow copy the answer is no. It is a deep copy (or perhaps copy on write) and will copy all the way down.
<?php
$a = ["meow", [1,2,3]];
$b = $a;
$a[1][] = 4;
echo json_encode($a), "\n";
# ["meow",[1,2,3,4]]
echo json_encode($b), "\n";
# ["meow",[1,2,3]]
There is however one specific exception to when something is copied and that is objects. Objects won’t be automatically copied like everything else.
<?php
class A {
public string $value;
public function __construct($in = "meow") {
$this->value = $in;
}
}
$a = new A;
$b = $a;
$a->value = "mew";
echo $a->value, "\n";
# mew
echo $b->value, "\n";
# mew
Now for the strangest one. Want to know to to copy an array in Erlang? You simply don’t. Everything is immutable in Erlang so once a variable has been assigned a value it is set in stone and can never be changed. The variable can have different values in different invocations of the same function but that is it. Erlang also has no normally shaped stack and no iteration so you can do infinite recursion using something called trampolines. Also the language syntax is a bit unusual which you can see below.
moan(A) -> io:fwrite("the value is ~w~n", [A]).
meow(B) ->
moan(B),
% the value is 1
A=B,
moan(A).
% the value is 1
main(_) ->
A=1,
meow(A).
The first thing we are going to do is make sure mod_header is enabled which can be done with the command
sudo a2enmod headers
and after that is done and the config is changed you should restart Apache with
sudo systemctl restart apache2
and we are ready for what config to be set.
Now we have the config which you should put in some file that Apache includes for example I have mine in /etc/apache2/sites-enabled/000-default.conf but you can put yours anywhere really. There are quite a few quirks in general we must defeat in order for Apache to just serve files as is without executing any of them by accident. This is the actual config I use to share some downloads between computers that as you can see very commented and explained.
<Directory /var/www/html/aria/finished>
# Various fixes
# Make sure everything is granted by default
# as certain file types might be limited by other config
<FilesMatch ".*">
Require all granted
</FilesMatch>
# .htaccess should not be read
AllowOverride None
# No symlinks here as they might lead to odd things unless you want them here ofc
Options -FollowSymLinks
Options -SymLinksIfOwnerMatch
# Turn off the feature of trying to correct faulty filenames to similar ones
Options -MultiViews
# Executable code
# No cgi scripts should be executed here
Options -ExecCGI
# Disable server side includes .shtml
Options -Includes
Options -IncludesNOEXEC
# Turn off php in this directory in a way that no .htaccess can turn them on again
<IfModule mod_php.c>
php_admin_flag engine off
</IfModule>
# Directory listings
Options +Indexes
# Good options for directory listing
IndexOptions +FancyIndexing +IgnoreCase +ShowForbidden
# Descending Date is good as the newest files
# will be in the top so you do not have to scroll
IndexOrderDefault Descending Date
# the README.html should not be automatically be read and shown as part of the listing
ReadmeName /
# The same but the HEADER.html that appears on top
HeaderName /
# Prevent index.html or similar from displaying and hiding the directory listing
DirectoryIndex disabled
# Content-Type
# Prevent text/html as that can run code we do not want to
# Many browsers will assume text/html if no Content-Type is found so we need to be explicit
# If there is no Content-Type header set to text/plain for viewing rather than download
Header set Content-Type "text/plain" "expr=-z %{CONTENT_TYPE}"
# If it is not a directory since those always end with slash
<FilesMatch "[^/]$">
# Set Content-Type to text/plain if it is set to text/html
# This means that only directory listings can output html
Header set Content-Type "text/plain" "expr=%{CONTENT_TYPE} =~ m|text/html|"
</FilesMatch>
</Directory>
# So there is a certain problem where IndexIgnoreReset does not work in a Directory
# section therefore defining a Location section instead is needed.
# If you wonder why there just cannot be a single Location section it is
# that FilesMatch and LocationMatch sections are forbidden inside Directory sections.
<Location "/aria/finished">
# Reset the index ignore so everything gets listed
IndexIgnoreReset ON
# Set the ignore to an impossible file to not ignore anything
IndexIgnore /
</Location>
Now we have finally defeated the web server and it will no longer do anything unexpected or strange hopefully.
]]>We are going to focus on Bash for this as there is dash, zsh and a bunch of others but what is inside bash is very similar to zsh and works almost identical except a few quirks. Most of these symbol related syntaxes are called expansions or substitutions.
To begin let’s define some variables to use to test these where one is a string, second one a simple array and third is resetting the shell input often called ARGV and is the arguments passed to the current script or function.
var='this is a variable with lowercase and UPPERCASE'
arr=('after ' ' before' ' surrounded ' 'alone')
set 'one ' ' two ' 'three'
Also we need to define how to actually display the output which we will do with this where lines starting with $ are commands being run.
$ printf %q\\n several "arguments here" to test ' this '
several
arguments\ here
to
test
\ this\
As it prints one thing per line escaped we can see what is actually shaped by these symbols. Now on to what ones there are.
Let’s get started with the magic symbols.
$ printf %q\\n "$@"
one\
\ two\
three
Good for getting all shell arguments to pass elsewhere.
$ printf %q\\n $*
one
two
three
$ set ' a ' 'b ' ' c d '
a
b
c
d
This does same as above but takes away spaces.
$ printf %q\\n "$*"
one\ \ \ two\ \ three
Yes this packs everything into a single argument.
$ printf %q\\n "$@"
one\
\ two\
three
$ printf %q\\n "${#var}"
47
Yes the length.
$ printf %q\\n "${#arr[@]}"
4
Yep the length.
printf %q\\n "${arr[@]}"
after\
\ before
\ surrounded\
alone
$ printf %q\\n ${arr[@]}
after
before
surrounded
alone
$ printf %q\\n "${arr[*]}"
after\ \ \ before\ \ surrounded\ \ alone
Good for joining arrays.
$ printf %q\\n ${arr[*]}
after\
\ before
\ surrounded\
alone
$ printf %q\\n ${arr[@]}
after\
\ before
\ surrounded\
alone
$ printf %q\\n "${arr[@]}"
after\
\ before
\ surrounded\
alone
$ printf %q\\n "${arr[@]:2}"
\ surrounded\
alone
$ printf %q\\n "${arr[@]:2:1}"
\ surrounded\
$ printf %q\\n ${arr[@]:1:1}
\ before
$ printf %q\\n ${arr[@]:3}
alone
$ printf %q\\n ${var#}
this\ is\ a\ variable\ with\ lowercase\ and\ UPPERCASE
$ printf %q\\n ${var#var}
this\ is\ a\ variable\ with\ lowercase\ and\ UPPERCASE
$ printf %q\\n ${var#*var}
iable\ with\ lowercase\ and\ UPPERCASE
$ printf %q\\n ${var#this}
\ is\ a\ variable\ with\ lowercase\ and\ UPPERCASE
$ printf %q\\n ${var#}
this\ is\ a\ variable\ with\ lowercase\ and\ UPPERCASE
$ printf %q\\n ${var#*var}
iable\ with\ lowercase\ and\ UPPERCASE
Quite convenient substring like feature.
$ printf %q\\n ${var%and*}
this\ is\ a\ variable\ with\ lowercase\
$ printf %q\\n ${var%with*}
this\ is\ a\ variable\
$ printf %q\\n ${var%CASE}
this\ is\ a\ variable\ with\ lowercase\ and\ UPPER
$ printf %q\\n ${var%low*}
this\ is\ a\ variable\ with\
$ printf %q\\n ${var% *}
this\ is\ a\ variable\ with\ lowercase\ and
Quite useful but might be hard to figure out
$ printf %q\\n ${var-default}
this\ is\ a\ variable\ with\ lowercase\ and\ UPPERCASE
$ printf %q\\n ${nonexistent-default}
default
Good when a variable might not be set.
printf %q\\n ${var:=default value}
this\ is\ a\ variable\ with\ lowercase\ and\ UPPERCASE
$ printf %q\\n ${varr:=default value}
default\ value
$ printf %q\\n ${varr}
default\ value
Similar to Python’s dict.setdefault.
$ printf %q\\n ${var:10}
variable\ with\ lowercase\ and\ UPPERCASE
$ printf %q\\n ${var:10:6}
variab
$ printf %q\\n ${var:1:10}
his\ is\ a\ v
Fully functional substring with a length argument that is better than those that want you to specify the end.
$ printf %q\\n ${var^^}
THIS\ IS\ A\ VARIABLE\ WITH\ LOWERCASE\ AND\ UPPERCASE
VERY!
$ printf %q\\n ${var,,}
this\ is\ a\ variable\ with\ lowercase\ and\ uppercase
no uppercase anymore.
$ printf %q\\n $(( 42 - 69 + 2 + 1337 ))
1312
Precisely what it says.
$ echo ${var@A}
var='this is a variable with lowercase and UPPERCASE'
$ echo ${arr[@]@A}
declare -a arr=([0]="after " [1]=" before" [2]=" surrounded " [3]="alone")
Notice the echo here but also yes this gives the array definition or string definition needed to properly export the variable
$ echo ${var@Q}
'this is a variable with lowercase and UPPERCASE'
$ echo ${arr[*]@Q}
'after ' ' before' ' surrounded ' 'alone'
$ echo ${arr[@]@Q}
'after ' ' before' ' surrounded ' 'alone'
This quotes stuff.
$ printf %q\\n ~- ~+ ~ $PWD $OLDPWD $HOME
/home
/home/pi
/home/pi
/home/pi
/home
/home/pi
This shows that shells do indeed have quite some features that can be used for quite a few tasks and while many of these tricks do not work in zsh it has its own magic.
]]>For this I recommend you use Mozilla Firefox Developer Edition and go into about:config in the address bar and set xpinstall.signatures.required to false or you can use Google Chrome. The reason for this is that if you use the standard Firefox it will not properly accept unsigned plugins or it might uninstall them whenever you restart Firefox.
Next there is a thing we must get to function and that is CORS which stands for Cross Origin Resource Sharing and that is a special set of headers to send client side HTTP requests inside a browser to a different domain than the page is on.
There is a in aria2.conf that will supposedly fix that
rpc-allow-origin-all=true
that is if it actually helped in this situation. I never got this to properly to work and it seems that one of the issues is that it really does try to use HTTPS at all times which breaks things. There are some things that should be possible in setting the setting rpc-secure to fix things but that requires many options to be set properly so we are doing it another way.
What we are going to do instead as a workaround is make an apache CORS proxy. Start by making sure you have apache installed which you can install by using
sudo apt install apache2
Next we must make sure certain mods are enabled with the following command
sudo a2enmod headers proxy_http ssl
Now outside any VirtualHost section we should set this and it can sort of be inside any .conf file in /etc/apache2/apache2.conf in the end
<Location /ariarpc>
ProxyPass http://127.0.0.1:6800/jsonrpc
Header setifempty Access-Control-Allow-Origin "*"
Header setifempty Access-Control-Allow-Headers "*"
</Location>
Now we must create a VirtualHost section for the SSL that the extension we are creating insists on using. It tends to exist at /etc/apache2/sites-enabled/000-default.conf so creating another virtual host there or in a file beside it might be good but ensure both virtual hosts exists where one has the content below.
<VirtualHost *:443>
SSLEngine on
SSLCertificateFile /etc/ssl/certs/ssl-cert-snakeoil.pem
SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
SSLCertificateChainFile /etc/ssl/certs/ssl-cert-snakeoil.pem
ServerName smol
ServerAdmin webmaster@localhost
DocumentRoot /var/www/html
<Directory /var/www/html>
AllowOverride All
</Directory>
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>
There is a risk that the selt signed cert is not there but you can generate it with the openssl command if needed.
Now we only need to restart apache and go to https://192.168.0.44/ariarpc or whatever the address is to your host and press to accept the fault certificate which we will need to have accepted for the extension to b e able to use it.
Now for the actual extension we need to create a zip file that has no subdirectories but does contain a few files called manifest.json, icon.png and background.js. For the icon just make anything really an image that is maybe 100x100 or so. In the manifest we can place the following to make sure the permissions are right and everything.
manifest.json
{
"background": {
"scripts": [
"background.js"
]
},
"action": {
"default_icon": "icon.png",
"default_title": "Share"
},
"browser_specific_settings": {
"gecko": {
"id": "downloadaddon@example.com"
}
},
"description": "Download Addon just for testing",
"icons": {
"128": "icon.png"
},
"manifest_version": 3,
"name": "Download Addon",
"permissions": [
"contextMenus",
"notifications"
],
"host_permissions": [
"<all_urls>"
],
"version": "1.0"
}
The actual script here will do the thing of creating a context menu with you can open with the second mouse button over links or images or pages and then it should send the request to your NAS or whatever has apache and aria2 on it but make sure to set the password and the address in the line with xhr.open in it.
background.js
function context(event, tab) {
var url = event.pageUrl;
if (event.menuItemId === "link") {
url = event.linkUrl;
} else if (event.menuItemId === "image") {
url = event.srcUrl
}
var data = {
"jsonrpc": "2.0",
"id": "whatever",
"method": "aria2.addUri",
"params": [`token:${token}`, [url]],
};
var xhr = new XMLHttpRequest();
xhr.onreadystatechange = function () {
if (xhr.readyState !== 4) return;
console.log(xhr.responseText);
chrome.notifications.create(null, {
"message": xhr.responseText,
"title": "Download",
"iconUrl": "icon.png",
"type": "basic",
})
};
xhr.open("POST", "http://192.168.0.44/ariarpc");
xhr.setRequestHeader("Content-Type", "application/json");
xhr.send(JSON.stringify(data));
}
browser.contextMenus.create({
"title": "Download page",
"id": "page",
"contexts": ["page"]
},
() => void browser.runtime.lastError,
);
browser.contextMenus.create({
"title": "Download link",
"id": "link",
"contexts": ["link"]
},
() => void browser.runtime.lastError,
);
browser.contextMenus.create({
"title": "Download image",
"id": "image",
"contexts": ["image"]
},
() => void browser.runtime.lastError,
);
browser.contextMenus.onClicked.addListener(context)
var token = "YourPasswordHerePleaseChange";
Now all of the 3 files should be inside the zip and now you are going to rename the zip file to anything with the .xpi ending and next you can run it inside Firefox Developer Edition by dragging it into tab or setting xpi files to run using it. Now go to any page and second clicking on anything at all like a link, an image or the page it self will give you a context menu to download it. Tell how your experience was in the comments if you decide to try this.
To solve this there was several things to consider. One thing was that if I moved the files that would be an issue since seeding. If I copied the files there was another issue of them taking up double the amount of space. However there is a third and even a fourth option. The third option is to make a symbolic link that acts just like that file in almost every single case except one where the files are moved or copied by a tool that is aware of those being links and then fail to copy the actual files. Therefore we are using a fourth option which is called a hard link.
A hard link is a very nifty feature where a file can exist in many places at once similarly to a value inside a programming language that is garbage collected. Similarly if all hard links to a file is removed the file is essentially gone from the file system and might be overwritten. Due to how this works if you create another link to the same file it will not take up all that files space and instead it just takes up a few bytes extra.
To create any form of link on a Linux system we will use the ln command which stands for link. The ln command has a bunch of flags like -f to forefully place a file destroying the previous file at target if necessary and -s which makes links symbolic. To use the command the first argument should be the path to the target source file which the newly created link should lead to and the second argument is where that link should be placed. The links can be relative paths or absolute but make sure you are specifying the correct path as of when the link is used as compared to when the link is created or just use absolute paths.
Back to what we need to do with aria for this. Let’s go to the home directory and make sure the directory .aria2 exists and inside it there should be a file called aria2.conf which you shoulc create if not existing.
/home/yeen/.aria2/aria2.conf
on-bt-download-complete=/home/yeen/.aria2/done.sh
on-download-complete=/home/yeen/.aria2/done.sh
dir=/home/yeen/Downloads/aria/unfinished
As you see my username is yeen but change it to what yours is. Also notice done.sh which we are also going to create which you can do with any text editor really but make sure you do chmod +x done.sh to make sure it is executable and here is the contents of it.
/home/yeen/.aria2/done.sh
#!/bin/bash
cd "$(dirname "$0")"
uid="$1"
code="$2"
file="$3"
srcdir='/home/yeen/Downloads/aria/unfinished/'
dstdir='/home/yeen/Downloads/aria/finished/'
# If file does not start with srcdir which
# would mean it is not inside the correct folder then exit
if [[ "$file" != "$srcdir"* ]]; then
exit 1
fi
# If $file is not a file then exit
if [[ ! -f "$file" ]]; then
exit 1
fi
# Get the relative path by subtracting the absolute directory path
rel="${file#$srcdir}"
echo $rel
# Get if the file is inside any kind of subdirectory then create it
dname="$(dirname "$rel")"
if [[ "$dname" != '.' ]]; then
mkdir -p "$dstdir$dname"
fi
# Hardlink since there is no s flag the file to the dest dir in the subdir if there
ln "$file" "$dstdir$rel"
There is quite a bunch of things inside here but all of it is to do the following steps.
So yeah that is not that complex and ln gets the arguments as absolute paths. Now there is just one step left that is to create a systemd unit to run the whole thing and let’s do a user unit for that. An easy way to create the unit is to run the command
systemctl --user edit --full --force aria
which will open nano or whatever editor you have chosen but if you have not selected an editor run select-editor first. Now for the contents to paste in
/home/yeen/.config/systemd/user/aria.service
[Unit]
Description=Aria2 download manager
[Service]
ExecStart=/bin/aria2c --enable-rpc --rpc-listen-all --rpc-secret YourPasswordHerePleaseChange
WorkingDirectory=/home/yeen/Downloads/aria/unfinished
[Install]
WantedBy=default.target
and save and exit.
Before you run it make sure all the paths are existing that you want to use for the downloads or else things might not work but you can run the following command to create all those directories
mkdir -p ~/.aria2/ ~/Downloads/aria2/{finished,unfinished}
Now you can start aria2 in RPC mode using this and it will run on every start and be ready for any downloads you add. Just run
systemctl --user enable --now aria
and it will start up now and be enabled on the next upstart.
There are a few more things that I recommend and one of them is to install webui-aria2 to be able to control aria since otherwise you will just have a JSON-RPC API to access it through.
Personally I use that and I also use my own browser extension which gives me a menu on every page to start a download which I might write about next.
Now if you get everything working you can just look at your finished files directory now and then and see whenever a file is downloaded and finished and start using it even tho the whole torrent that is downloading is not fully finished yet or see every other file too as soon as it is downloaded without it taking extra hard drive space as soon as it finished.
]]>Escaping code that is generated somehow is a extremely important practice. SQL injections are a huge issue when it comes to code that is not properly escaped therefore you should use prepared statements whenever you can in SQL. In this article however we are going to look at places where the code is escaped.
There are various examples of this but the definition of it would be that you have any form of data structure that is usually a string and want to fit it inside another data structure. Things such as numbers tend to not be needed to be escaped as most languages use the same format for numbers but it is a good practice to do it anyway. You could call this a form of code generation as the goal is to produce a textual representation of the data inside the target language as a literal.
Now the goal is to produce something that does not break the syntax of the target language. While prepending and appending double quote " might work many times to escape something it does not work on something actually containing double quote and while prepending \ to each double quote might work in those cases it might not be the best representation of the string.
Due to the nature of this there might be multiple representation of the same string such as some strings contain characters that might or might not be escaped such as newlines that might either be an actual newline written in the code or the escaped version \n.
HTML is escaped by first replacing & with with & or & then < with < or < and > with > or >. The characters &, < and > have special meaning in HTML for example a HTML tag looks like <tag></tag>. XML can have some additional restrictions of what is allowed and there might be other additional restrictions on both XML and HTML when it comes to unprintable characters.
Attributes in HTML have additional restrictions of what characters need to be escaped. It also has some characters that technically no longer needs to be escaped like < and > but " needs to be escaped into " or " and ' into ' or '. There are esceptions and special cases to this but it is better to escape than not escape to prevent accidents.
It is possible to escape characters that you do not need to escape for readability reasons such as replacing \n an actual newline with . An example of this is that you can write
<meta name="og:description" content="something
with
lines" />
and it will parse just fine but you should really write
<meta name="og:description" content="something with lines" />
if you want some better looking indentation. As you see is a newline since it is of the value 10 in base 10 which is a newline in encodings such as ASCII and UTF-8.
See html.escape to escape HTML in Python.
URLs use something called URL encoding to escape characters that have a special meaning inside URLs or in application/x-www-form-urlencoded HTTP requests. URL encoding works that all characters that have special meaning or is not printable are replaced with a percent sign % and two hex characters which means that it uses an alphabet not only 0-9 but also A-Z. This means that newline encoded is %0A rather than as it is in HTML as the number base is 16 and not 10.
The characters that should be escaped in an URL is both the non printable characters which are under 32 in ASCII and UTF-8 and above 126 in ASCII and the following printable chars.
@&*][,"|!(' #;<^=`+%$)\}:{>?/
# Which escapes into
%40%26%2A%5D%5B%2C%22%7C%21%28%27%20%23%3B%3C%5E%3D%60%2B%25%24%29%5C%7D%3A%7B%3E%3F%2F
# Which is 3 times bigger
There are more things to consider for example a special rule that says that a space URL in a browser might either be written as + as plus %20 or an actual space ` ` which will be converted into %20 unlike the +. Certain APIs might have strange rules of when to use plus and when to use + or %20 such as the Telldus API in the past.
There are some other oddities such as / is not escaped by Python’s quote but it is using quote_plus and I have no idea why but found out when checking the Python source code for this article.
As always it is possible to escape characters that do not need to be escaped like you can escape the entirety of the alphabet in uppercase which looks like this
%41%42%43%44%45%46%47%48%49%4a%4b%4c%4d%4e%4f%50%51%52%53%54%55%56%57%58%59%5a
which could maybe be good if you want to hide some code so you accidentally do not remember it or something like that or maybe you are trying to sneak past some code past some anti cheat or antivirus program.
You can use the following function to completely URL encode any string in Python.
def escall(i): return ''.join('%%%02x' % ord(a) for a in i)
See urllib.quote to escape URLs in Python.
JSON which is the object notation for JavaScript used for data transport is quite simple that it is based on C and has similar escape codes and it is very well documented. It is valid code in JavaScript and sort of valid Python. There is a funny trick you can do in Python where you enter the line
null,false,true=None,False,True
and after it is in scope you can usually just eval to get the JSON out which is something you should really not do with untrusted data.
JSON uses unicode escape codes for escaping so if we make an example function to escape it would look like this
def escall2(i): return ''.join('\\u%04x' % ord(a) for a in i)
and would escape every single character to a string in JSON.
Anyway this some basic examples how escaping in programming languages work and I hope you had fun reading.
]]>Known falsehoods: false, Boolean(false)
Java has the most simple rule that only a boolean may ever be considered considered for truth or falsehood and that is either the simple boolean primitive true or the object Boolean() that is. Be careful that you do not dereference a null pointer as a variable that is set as the type Boolean can contain one unlike the primitive.
Known falsehoods: False, None, "", b"", 0, 0.0, (), {}, [], set()
Python has way more complex rules when it comes to what is True and False and yes they must be capitalized in order to work. Empty strings are considered false and also empty bytestrings bytes are also considered false. Numbers like the int 0 is considered false and the float 0 is also considered false and similarly with the complex 0j and so on.
What is unusual with Python here is that any type can chose if it is True or False at any time using the __bool__ method that is called whenever there is a comparison so () tuples set() sets [] lists and {} dicts are considered false if empty.
If you define your own type you can do like this
class A:
def __bool__(self):
return False
print(bool(A()))
And it will print False in this example as that is what the type compares to.
Known falsehoods: 0, "0", "", undef
Perl has some sort of consistent rules except one single bizarre quirk when it comes to that a string containing exactly a single 0 zero character will evaluate to false. However in Perl there is not a real boolean type in all versions. I am very new to this language so there might be a lot more quirks like these are the types are not acting like I am used to
Known falsehoods: false, null, 0, 0.0, "", "0", []
Just like Perl the string containing exactly one zero is false so you should probably compare against what you really want to compare against which you should do in code anyway.
Keep in mind that there is also the special function isset that does a different kind of truth check compared to boolval in that everything that is not null or false is considered true.
isset also has the functionality that you can use isset($array["nonexistentkey"]) without getting any warnings or errors so you do not have to use in_array("nonexistentkey", $array, true) for example.
There is “supposed” to be boolean overloading for classes in PHP but that is a whole other absurdity.
Known falsehoods: false, undefined, 0, ""
JavaScript has simpler rules than you would expect from such a wonky language but just like PHP it has extremely wonky rules for comparison that I might write about another time.
To cast to boolean in PHP you can use !!something or Boolean(something) but you are not supposed to use new Boolean(something) for some reason.
Anyway it is good to know how to really check if something is really false in programming before you assume that something is considered in when compared.
]]>At first we are going to start with how to actually set them up and how they work.
The first thing to do is to create the socket unit which looks like this
testsocket.socket
[Unit]
Description=Test socket
[Socket]
ListenStream=1024
Accept=yes
[Install]
WantedBy=default.target
To create a unit you can use the command
systemctl --user edit --full --force testsocket.socket # For user unit
sudo systemctl edit --full --force testsocket.socket # For system unit
Which will open up an editor like nano but if you have not selected an editor I recommend using nano which you can set using
select-editor
Except this unit you also need to define a second unit that is a service
testsocket@.service
[Unit]
Description=Test socket service
[Service]
ExecStart=/home/yeen/blep.sh
And do note that there is an @ in the name just like in the Failure handler guide from a while ago.
Not to take a look what the started script actually does using lsof -p $$ inside the script. As you can probably see a socket is open in file descriptor 3 and I have no idea what the pipe in 4 is but that does not matter much.
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
blep.sh 9877 yeen cwd DIR 8,2 4096 50298626 /home/yeen
blep.sh 9877 yeen rtd DIR 8,2 4096 2 /
blep.sh 9877 yeen txt REG 8,2 125688 262951 /usr/bin/dash
blep.sh 9877 yeen mem REG 8,2 2220400 263025 /usr/lib/x86_64-linux-gnu/libc.so.6
blep.sh 9877 yeen mem REG 8,2 240936 263020 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
blep.sh 9877 yeen 0r CHR 1,3 0t0 5 /dev/null
blep.sh 9877 yeen 1u unix 0x0000000000000000 0t0 120353 type=STREAM
blep.sh 9877 yeen 2u unix 0x0000000000000000 0t0 120353 type=STREAM
blep.sh 9877 yeen 3u IPv6 121878 0t0 TCP localhost:1024->localhost:42958 (ESTABLISHED)
blep.sh 9877 yeen 4r FIFO 0,13 0t0 117586 pipe
blep.sh 9877 yeen 10r REG 8,2 81 50333753 /home/yeen/blep.sh
The script itself used to test it is here
``blep.sh`
#!/bin/sh
a="$(lsof -p $$)"
echo "$a" > ~/sdfdfg.txt
echo "$a" >&3
env > env.txt
What the >&3 does is tell it to write to file descriptor 3 so the lsof output from above is emitted from netcat connecting to the systemd socket.
The reason for env to be there is that some variables set there might be very useful and we can take a look at them here filtered from what usually exists in the env vars of systemd units being run.
12,13c12,13
< INVOCATION_ID=2106c6f489634605b77ec4c53b18e6ad
< JOURNAL_STREAM=8:139232
---
> INVOCATION_ID=10f97a5493074bf8b914b8bbbcd421c2
> JOURNAL_STREAM=8:120353
23a24,26
> LISTEN_FDNAMES=connection
> LISTEN_FDS=1
> LISTEN_PID=9877
29a33,34
> REMOTE_ADDR=127.0.0.1
> REMOTE_PORT=42958
32c37
< SHLVL=1
---
> SHLVL=0
34c39
< SYSTEMD_EXEC_PID=11197
---
> SYSTEMD_EXEC_PID=9877
36c41
< _=/usr/bin/env
---
> _=/usr/bin/dbus-update-activation-environment
As you see there are some things to take note of like REMOTE_ADDR=127.0.0.1 and REMOTE_PORT=42958 which were from a run where I just ran nc localhost 1024 which is netcat.
If you wonder how to get this diff do something like
systemd-run --user bash -c 'env > enva.txt'
diff <(sort enva.txt) <(sort env.txt)
And you will get a good diff.
Now we can test with python too by making a script in it that receives the invocation.
blep.py
#!/usr/bin/python3
import socket
s = socket.fromfd(3, socket.AF_INET, socket.SOCK_STREAM, proto=0)
s.send(b"Blep I am cat meow\n")
s.close()
And as you can guess it will send Blep I am cat meow to anyone who connects to the socket.
One last note is that you might have to use a command like
systemctl --user status testsocket@\*
and scroll a bit up and down to find the status as it creates a temporary service for every invocation that you can find in the logs.
Anyway now you know the basics of systemd sockets so you can have fun with them.
So apparently you can set stdin and stdout to be used to write to the socket. Here is an example. Start by changing your
testsocket@.service
[Unit]
Description=Test socket
[Service]
ExecStart=/home/yeen/blep.sh
StandardInput=socket
StandardOutput=socket
StandardError=journal
It is possible to set all 3 to socket but you should generally set the error one specifically to journal since otherwise it will write stderr to the socket too instead of into the journal.
Now we can update the
blep.sh
#!/bin/sh
lsof -p $$
echo log something here >&2
And now you can log things while also using stdout to write to the socket.
It is also very useful to start another terminal and use the following command to follow everything that happens in the log.
journalctl --user -f
This will bring up the task manager directly which is incredibly useful.
This will bring up the start menu which is good when you have a non working windows key such as when you have removed it for gaming purposes.
This will bring up a menu where you take a screenshot just by clicking on a window and you can also select other modes using the menu that pops up.
This will use Windows Game Bar to take a screenshot which is good for games that are not on Steam therefore does not have any F12 screenshot key.
This will bring up Windows gamebar where you can do things like record video of games, adjust volume controls or take screenshots.
This will take the current window and take a screenshot of it and place in clipboard. I useed to use this a lot before I knew of other methods and had a special tool that I ran afterwards to save the image. Then I made an even better tool for it. Keep in mind if Dropbox is installed it tends to use this shortcut to save a screenshot to Dropbox.
This will bring up a menu for the current window where you can for example maximize or minimize or even move it. This is very good when windows get stuck in weird positions.
This will scroll up and down without moving the cursor
Skip a whole word forwards or backwards in a text document. You can also hold down shift to select at the same time
This will indent the lines where the text is selected one step and you can use shift also at the same time to unindent instead.
This will bring up the task manager for firefox which can be very
Just pressing space bar without anything selected will scroll down. If you use shift at the same time you will scroll up instead.
This will switch to the next tab and holding shift at the same time will go to the previous.
Will switch to the tab with that position.
Open a special take screenshot menu with built in cropping.
Firefox has a large amount of shortcuts in general which are good to know
Which of these shortcuts did you already know and do you know of any other useful shortcuts. Tell in the comments.
]]>authorized_keys or anything like that really. It is quite common to do this manually by opening the file and putting in the line if it is missing and so on. You can however automate the entire process using shell.
I went through a lot of iterations in order to set this up as the compatibilities between shells are not the best. Here you can see the actual result of what I came up wit in order to solve the problem and make a script to ensure that the keys are in the correct location and installed.
ensurefileline() {
file="$1"
line="$2"
if [ ! -f "$file" ]; then
touch "$file"
echo creating file >&2
echo "$line" >> "$file"
else
echo file exists
if grep -F "$line" "$file" >/dev/null 2>&1 ; then
echo line in file >&2
else
echo no line in file >&2
echo "$line" >> "$file"
fi
fi
}
keys=$(
cat << 'EOT'
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCLFZusq0Gmyk17xRI0b8vXalKM9cKyjcA6SCSaHoGYHlz39S32zZ7DacDHYx1txiglTEIUVgjzIah2gWcpB7FFvIKoWvfF5VVXhleUjd3y8erRPJ+TuPV5f6po5eUL1iqtTEVrbRmIExzKVPQRSFRuYuCinY6P+wEDm902Kql/aNl9YqPRlc6pnYIvnyXVw2HeyR23noro8GV7CKjBw4PfbWDmH52CYDmvCs0W8xwl27Vg1N97wwMNe93rCqax928nZBdSNipo1bVmL9MhToJgkIMdF73+aNWV4/ug60E79PcB4l6PwnxT2NPASgkcTlugsrkonEfI9xy4OIZt0oD75RQWkbTLU6/di9l8Te/rcQv2UZQEEhU+DesL6nZWRo8YuPbo0WcMcDczXOHq/nsI4vm4rghdR/NotalZxbClpXwAx4tG+mf913fSNHCbnORAUrPMBN9l1XavuVyITvTNRlFCZGNsGXOvH44jpiedU9vSX+kGGofBp+QrDYmjaGs= root@clear
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLBCvhDnR2oEuFGaLxW+63HYotCt2wIWEB36nXeeuPznoT1TEcBuYT0XsGxA7jzr48X4yHzU38TaHXHAghknMiSNmZpcNxNc49jshj7DyFoeOa84O1cq9An1oVODfOkiQ9Q8k1p4nG+dPnxlBBGXYoINl4qbdZM0NEJM0pXGtXdz5BtGBcL20ZsfJZNHp2jDFtGLxn6TY04wh6KMZ4tHORgVQdMboHKvrUUxtH+s/Y47MAPiwJqbXfCOkZb2dLhrYfV6RhbDZ3Bo3CR3oKtqhBsuBn6HK7FXHHvwzIkuYxt0MowV4/6UO6AIaGC9sFNqkzCyj1diN/rt8UBRC/gNz+2eRsl7pVIt1WBty7XY7Nq9Q5Vo2IjZ/M0k2HY8lLQUxiKzTw1eTYEfqbg9pZzs7ljG9f1ohKcQ1+4WZWh9w52jovdMjB8qvfQLltRyoPEl9xOk3v3+UGCJM0aJTPviGBi+Io0qlNfdzkqCqYzS5ETFCF/PH4cSecokOWALiN0A0= root@clear
EOT
)
OLDIFS="$IFS" # Save IFS in case
export IFS="
" # This looks cursed but does give compatibility with /bin/sh
for key in $( echo "$keys" ); do # This iterates and splits by IFS
ensurefileline ~/.ssh/authorized_keys "$key"
done
IFS="$OLDIFS" # Restore IFS
Debugging this entire thing was quite a process as the entire thing inside the function was no problem as it ran no problem inside all shells but some of the other stuff had huge issues. To get exactly the right heredoc syntax was a bit of a pain until I found the 'EOT' thing that works well and to enclose it all in a command substitution with $() in order to get it into a variable.
Here we are going to look at some strange quirks in the whole thing.
This is looking very odd but it works in both zsh, sh and bash
IFS="
"
I have no idea if there is a better solution for this.
However there are other syntaxes that work on many shells such as the following
variable=$'\n'
But the result of that code running in all shells is the following
test.sh
# sh
['$\\n']
# zsh
['\n']
# bash
['\n']
Here as you can see the content of the variable becomes $\\n which is a literal dollar sign, a backslash and the character n where there is supposed to be a newline.
Here is what is used to test that and get that output. To test this set this up to exist in PATH as args
args
import sys
print(sys.argv[1:])
Ensure test.sh exist with the contents to test and run this in command line
echo '#' sh
sh test.sh
echo '#' zsh
zsh test.sh
echo '#' bash
bash test.sh
First to give an idea of this we are going to put that args command before the ensurefileline function call to see what is being outputted and run first like normal to see what it should look like.
# sh
['ensurefileline', '/home/pi/.ssh/authorized_keys', 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCLFZusq0Gmyk17xRI0b8vXalKM9cKyjcA6SCSaHoGYHlz39S32zZ7DacDHYx1txiglTEIUVgjzIah2gWcpB7FFvIKoWvfF5VVXhleUjd3y8erRPJ+TuPV5f6po5eUL1iqtTEVrbRmIExzKVPQRSFRuYuCinY6P+wEDm902Kql/aNl9YqPRlc6pnYIvnyXVw2HeyR23noro8GV7CKjBw4PfbWDmH52CYDmvCs0W8xwl27Vg1N97wwMNe93rCqax928nZBdSNipo1bVmL9MhToJgkIMdF73+aNWV4/ug60E79PcB4l6PwnxT2NPASgkcTlugsrkonEfI9xy4OIZt0oD75RQWkbTLU6/di9l8Te/rcQv2UZQEEhU+DesL6nZWRo8YuPbo0WcMcDczXOHq/nsI4vm4rghdR/NotalZxbClpXwAx4tG+mf913fSNHCbnORAUrPMBN9l1XavuVyITvTNRlFCZGNsGXOvH44jpiedU9vSX+kGGofBp+QrDYmjaGs= root@clear']
['ensurefileline', '/home/pi/.ssh/authorized_keys', 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLBCvhDnR2oEuFGaLxW+63HYotCt2wIWEB36nXeeuPznoT1TEcBuYT0XsGxA7jzr48X4yHzU38TaHXHAghknMiSNmZpcNxNc49jshj7DyFoeOa84O1cq9An1oVODfOkiQ9Q8k1p4nG+dPnxlBBGXYoINl4qbdZM0NEJM0pXGtXdz5BtGBcL20ZsfJZNHp2jDFtGLxn6TY04wh6KMZ4tHORgVQdMboHKvrUUxtH+s/Y47MAPiwJqbXfCOkZb2dLhrYfV6RhbDZ3Bo3CR3oKtqhBsuBn6HK7FXHHvwzIkuYxt0MowV4/6UO6AIaGC9sFNqkzCyj1diN/rt8UBRC/gNz+2eRsl7pVIt1WBty7XY7Nq9Q5Vo2IjZ/M0k2HY8lLQUxiKzTw1eTYEfqbg9pZzs7ljG9f1ohKcQ1+4WZWh9w52jovdMjB8qvfQLltRyoPEl9xOk3v3+UGCJM0aJTPviGBi+Io0qlNfdzkqCqYzS5ETFCF/PH4cSecokOWALiN0A0= root@clear']
# zsh
['ensurefileline', '/home/pi/.ssh/authorized_keys', 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCLFZusq0Gmyk17xRI0b8vXalKM9cKyjcA6SCSaHoGYHlz39S32zZ7DacDHYx1txiglTEIUVgjzIah2gWcpB7FFvIKoWvfF5VVXhleUjd3y8erRPJ+TuPV5f6po5eUL1iqtTEVrbRmIExzKVPQRSFRuYuCinY6P+wEDm902Kql/aNl9YqPRlc6pnYIvnyXVw2HeyR23noro8GV7CKjBw4PfbWDmH52CYDmvCs0W8xwl27Vg1N97wwMNe93rCqax928nZBdSNipo1bVmL9MhToJgkIMdF73+aNWV4/ug60E79PcB4l6PwnxT2NPASgkcTlugsrkonEfI9xy4OIZt0oD75RQWkbTLU6/di9l8Te/rcQv2UZQEEhU+DesL6nZWRo8YuPbo0WcMcDczXOHq/nsI4vm4rghdR/NotalZxbClpXwAx4tG+mf913fSNHCbnORAUrPMBN9l1XavuVyITvTNRlFCZGNsGXOvH44jpiedU9vSX+kGGofBp+QrDYmjaGs= root@clear']
['ensurefileline', '/home/pi/.ssh/authorized_keys', 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLBCvhDnR2oEuFGaLxW+63HYotCt2wIWEB36nXeeuPznoT1TEcBuYT0XsGxA7jzr48X4yHzU38TaHXHAghknMiSNmZpcNxNc49jshj7DyFoeOa84O1cq9An1oVODfOkiQ9Q8k1p4nG+dPnxlBBGXYoINl4qbdZM0NEJM0pXGtXdz5BtGBcL20ZsfJZNHp2jDFtGLxn6TY04wh6KMZ4tHORgVQdMboHKvrUUxtH+s/Y47MAPiwJqbXfCOkZb2dLhrYfV6RhbDZ3Bo3CR3oKtqhBsuBn6HK7FXHHvwzIkuYxt0MowV4/6UO6AIaGC9sFNqkzCyj1diN/rt8UBRC/gNz+2eRsl7pVIt1WBty7XY7Nq9Q5Vo2IjZ/M0k2HY8lLQUxiKzTw1eTYEfqbg9pZzs7ljG9f1ohKcQ1+4WZWh9w52jovdMjB8qvfQLltRyoPEl9xOk3v3+UGCJM0aJTPviGBi+Io0qlNfdzkqCqYzS5ETFCF/PH4cSecokOWALiN0A0= root@clear']
# bash
['ensurefileline', '/home/pi/.ssh/authorized_keys', 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCLFZusq0Gmyk17xRI0b8vXalKM9cKyjcA6SCSaHoGYHlz39S32zZ7DacDHYx1txiglTEIUVgjzIah2gWcpB7FFvIKoWvfF5VVXhleUjd3y8erRPJ+TuPV5f6po5eUL1iqtTEVrbRmIExzKVPQRSFRuYuCinY6P+wEDm902Kql/aNl9YqPRlc6pnYIvnyXVw2HeyR23noro8GV7CKjBw4PfbWDmH52CYDmvCs0W8xwl27Vg1N97wwMNe93rCqax928nZBdSNipo1bVmL9MhToJgkIMdF73+aNWV4/ug60E79PcB4l6PwnxT2NPASgkcTlugsrkonEfI9xy4OIZt0oD75RQWkbTLU6/di9l8Te/rcQv2UZQEEhU+DesL6nZWRo8YuPbo0WcMcDczXOHq/nsI4vm4rghdR/NotalZxbClpXwAx4tG+mf913fSNHCbnORAUrPMBN9l1XavuVyITvTNRlFCZGNsGXOvH44jpiedU9vSX+kGGofBp+QrDYmjaGs= root@clear']
['ensurefileline', '/home/pi/.ssh/authorized_keys', 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLBCvhDnR2oEuFGaLxW+63HYotCt2wIWEB36nXeeuPznoT1TEcBuYT0XsGxA7jzr48X4yHzU38TaHXHAghknMiSNmZpcNxNc49jshj7DyFoeOa84O1cq9An1oVODfOkiQ9Q8k1p4nG+dPnxlBBGXYoINl4qbdZM0NEJM0pXGtXdz5BtGBcL20ZsfJZNHp2jDFtGLxn6TY04wh6KMZ4tHORgVQdMboHKvrUUxtH+s/Y47MAPiwJqbXfCOkZb2dLhrYfV6RhbDZ3Bo3CR3oKtqhBsuBn6HK7FXHHvwzIkuYxt0MowV4/6UO6AIaGC9sFNqkzCyj1diN/rt8UBRC/gNz+2eRsl7pVIt1WBty7XY7Nq9Q5Vo2IjZ/M0k2HY8lLQUxiKzTw1eTYEfqbg9pZzs7ljG9f1ohKcQ1+4WZWh9w52jovdMjB8qvfQLltRyoPEl9xOk3v3+UGCJM0aJTPviGBi+Io0qlNfdzkqCqYzS5ETFCF/PH4cSecokOWALiN0A0= root@clear']
This looks correct and you know what the function is supposed to be called with but now look at this.
# sh
['ensurefileline', '/home/pi/.ssh/authorized_keys', 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCLFZusq0Gmyk17xRI0b8vXalKM9cKyjcA6SCSaHoGYHlz39S32zZ7DacDHYx1txiglTEIUVgjzIah2gWcpB7FFvIKoWvfF5VVXhleUjd3y8erRPJ+TuPV5f6po5eUL1iqtTEVrbRmIExzKVPQRSFRuYuCinY6P+wEDm902Kql/aNl9YqPRlc6pnYIvnyXVw2HeyR23noro8GV7CKjBw4PfbWDmH52CYDmvCs0W8xwl27Vg1N97wwMNe93rCqax928nZBdSNipo1bVmL9MhToJgkIMdF73+aNWV4/ug60E79PcB4l6PwnxT2NPASgkcTlugsrkonEfI9xy4OIZt0oD75RQWkbTLU6/di9l8Te/rcQv2UZQEEhU+DesL6nZWRo8YuPbo0WcMcDczXOHq/nsI4vm4rghdR/NotalZxbClpXwAx4tG+mf913fSNHCbnORAUrPMBN9l1XavuVyITvTNRlFCZGNsGXOvH44jpiedU9vSX+kGGofBp+QrDYmjaGs= root@clear']
['ensurefileline', '/home/pi/.ssh/authorized_keys', 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLBCvhDnR2oEuFGaLxW+63HYotCt2wIWEB36nXeeuPznoT1TEcBuYT0XsGxA7jzr48X4yHzU38TaHXHAghknMiSNmZpcNxNc49jshj7DyFoeOa84O1cq9An1oVODfOkiQ9Q8k1p4nG+dPnxlBBGXYoINl4qbdZM0NEJM0pXGtXdz5BtGBcL20ZsfJZNHp2jDFtGLxn6TY04wh6KMZ4tHORgVQdMboHKvrUUxtH+s/Y47MAPiwJqbXfCOkZb2dLhrYfV6RhbDZ3Bo3CR3oKtqhBsuBn6HK7FXHHvwzIkuYxt0MowV4/6UO6AIaGC9sFNqkzCyj1diN/rt8UBRC/gNz+2eRsl7pVIt1WBty7XY7Nq9Q5Vo2IjZ/M0k2HY8lLQUxiKzTw1eTYEfqbg9pZzs7ljG9f1ohKcQ1+4WZWh9w52jovdMjB8qvfQLltRyoPEl9xOk3v3+UGCJM0aJTPviGBi+Io0qlNfdzkqCqYzS5ETFCF/PH4cSecokOWALiN0A0= root@clear']
# zsh
['ensurefileline', '/home/pi/.ssh/authorized_keys', 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCLFZusq0Gmyk17xRI0b8vXalKM9cKyjcA6SCSaHoGYHlz39S32zZ7DacDHYx1txiglTEIUVgjzIah2gWcpB7FFvIKoWvfF5VVXhleUjd3y8erRPJ+TuPV5f6po5eUL1iqtTEVrbRmIExzKVPQRSFRuYuCinY6P+wEDm902Kql/aNl9YqPRlc6pnYIvnyXVw2HeyR23noro8GV7CKjBw4PfbWDmH52CYDmvCs0W8xwl27Vg1N97wwMNe93rCqax928nZBdSNipo1bVmL9MhToJgkIMdF73+aNWV4/ug60E79PcB4l6PwnxT2NPASgkcTlugsrkonEfI9xy4OIZt0oD75RQWkbTLU6/di9l8Te/rcQv2UZQEEhU+DesL6nZWRo8YuPbo0WcMcDczXOHq/nsI4vm4rghdR/NotalZxbClpXwAx4tG+mf913fSNHCbnORAUrPMBN9l1XavuVyITvTNRlFCZGNsGXOvH44jpiedU9vSX+kGGofBp+QrDYmjaGs= root@clear\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLBCvhDnR2oEuFGaLxW+63HYotCt2wIWEB36nXeeuPznoT1TEcBuYT0XsGxA7jzr48X4yHzU38TaHXHAghknMiSNmZpcNxNc49jshj7DyFoeOa84O1cq9An1oVODfOkiQ9Q8k1p4nG+dPnxlBBGXYoINl4qbdZM0NEJM0pXGtXdz5BtGBcL20ZsfJZNHp2jDFtGLxn6TY04wh6KMZ4tHORgVQdMboHKvrUUxtH+s/Y47MAPiwJqbXfCOkZb2dLhrYfV6RhbDZ3Bo3CR3oKtqhBsuBn6HK7FXHHvwzIkuYxt0MowV4/6UO6AIaGC9sFNqkzCyj1diN/rt8UBRC/gNz+2eRsl7pVIt1WBty7XY7Nq9Q5Vo2IjZ/M0k2HY8lLQUxiKzTw1eTYEfqbg9pZzs7ljG9f1ohKcQ1+4WZWh9w52jovdMjB8qvfQLltRyoPEl9xOk3v3+UGCJM0aJTPviGBi+Io0qlNfdzkqCqYzS5ETFCF/PH4cSecokOWALiN0A0= root@clear']
# bash
['ensurefileline', '/home/pi/.ssh/authorized_keys', 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCLFZusq0Gmyk17xRI0b8vXalKM9cKyjcA6SCSaHoGYHlz39S32zZ7DacDHYx1txiglTEIUVgjzIah2gWcpB7FFvIKoWvfF5VVXhleUjd3y8erRPJ+TuPV5f6po5eUL1iqtTEVrbRmIExzKVPQRSFRuYuCinY6P+wEDm902Kql/aNl9YqPRlc6pnYIvnyXVw2HeyR23noro8GV7CKjBw4PfbWDmH52CYDmvCs0W8xwl27Vg1N97wwMNe93rCqax928nZBdSNipo1bVmL9MhToJgkIMdF73+aNWV4/ug60E79PcB4l6PwnxT2NPASgkcTlugsrkonEfI9xy4OIZt0oD75RQWkbTLU6/di9l8Te/rcQv2UZQEEhU+DesL6nZWRo8YuPbo0WcMcDczXOHq/nsI4vm4rghdR/NotalZxbClpXwAx4tG+mf913fSNHCbnORAUrPMBN9l1XavuVyITvTNRlFCZGNsGXOvH44jpiedU9vSX+kGGofBp+QrDYmjaGs= root@clear']
['ensurefileline', '/home/pi/.ssh/authorized_keys', 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLBCvhDnR2oEuFGaLxW+63HYotCt2wIWEB36nXeeuPznoT1TEcBuYT0XsGxA7jzr48X4yHzU38TaHXHAghknMiSNmZpcNxNc49jshj7DyFoeOa84O1cq9An1oVODfOkiQ9Q8k1p4nG+dPnxlBBGXYoINl4qbdZM0NEJM0pXGtXdz5BtGBcL20ZsfJZNHp2jDFtGLxn6TY04wh6KMZ4tHORgVQdMboHKvrUUxtH+s/Y47MAPiwJqbXfCOkZb2dLhrYfV6RhbDZ3Bo3CR3oKtqhBsuBn6HK7FXHHvwzIkuYxt0MowV4/6UO6AIaGC9sFNqkzCyj1diN/rt8UBRC/gNz+2eRsl7pVIt1WBty7XY7Nq9Q5Vo2IjZ/M0k2HY8lLQUxiKzTw1eTYEfqbg9pZzs7ljG9f1ohKcQ1+4WZWh9w52jovdMjB8qvfQLltRyoPEl9xOk3v3+UGCJM0aJTPviGBi+Io0qlNfdzkqCqYzS5ETFCF/PH4cSecokOWALiN0A0= root@clear']
Can you see the difference? If you do not see it did not split the content inside the heredoc by newline. The line
for key in $( echo "$keys" ); do
Was replaced with
for key in $keys; do
Which looks like it makes sense but this does not function inside zsh for some reason. That reason is zsh splitting does not work like normal and according to Stack Exchange in Zsh command expansions are split but not variable expansion.
So this goal of the entire this as discussed in the beginning is to be able to change some configs to ensure consistency and such. Lets give another example of usage where we have a file that is ~/.aliases and we want it sourced and we want to make sure it is sourced but only once on multiple executions of some install script. We create the following code for that.
ensurefileline .zshrc "if [ -f ~/.aliases ]; then source ~/.aliases fi"
ensurefileline .bashrc "if [ -f ~/.aliases ]; then . ~/.aliases fi"
This can then be run any number of times and only if the line is missing it will be added to the file. Another thing to note is that the grep command must be called with the -F flag that uses a fixed string as otherwise we would have to escape it somehow. Try this out if you want and have fun