GrooveStomp

GrooveStomp Profile Picture

PostgreSQL Password Encryption

I’m working on a side project to gain more exposure to different kinds of projects in Go. This one is a pretty typical web application involving multiple authenticated users. For data storage I’m using PostgreSQL for it’s UUID type and cryptographic extensions. The cryptographic extensions are interesting, because of things like chkpass1.

My project right now is using chkpass for storing user passwords. Almost immediately after hooking this up I started wondering how chkpass actually works. Let’s take a look.

The encryption uses the standard Unix function crypt(), and so it suffers from all the usual limitations of that function; notably that only the first eight characters of a password are considered.

So already we see something special:

only the first eight characters of a password are considered

Interesting!

And let’s look at the documentation for crypt2:

key is a user’s typed password.

salt is a two-character string chosen from the set [a-zA-Z0-9./]. This string is used to perturb the algorithm in one of 4096 different ways.

By taking the lowest 7 bits of each of the first eight characters of the key, a 56-bit key is obtained. This 56-bit key is used to encrypt repeatedly a constant string (usually a string consisting of all zeros). The returned value points to the encrypted password, a series of 13 printable ASCII characters (the first two characters represent the salt itself). The return value points to static data whose content is overwritten by each call.

Warning: the key space consists of 2**56 equal 7.2e16 possible values. Exhaustive searches of this key space are possible using massively parallel computers. Software, such as crack(1), is available which will search the portion of this key space that is generally used by humans for passwords. Hence, password selection should, at minimum, avoid common words and names. The use of a passwd(1) program that checks for crackable passwords during the selection process is recommended.

The DES algorithm itself has a few quirks which make the use of the crypt() interface a very poor choice for anything other than password authentication. If you are planning on using the crypt() interface for a cryptography project, don’t do it: get a good book on encryption and one of the widely available DES libraries.

There’s a lot there that’s good to know. A short summary here based on my reading:

This is very neat, but reading this makes me lean towards not using it for password storage, especially with statements like:

If you are planning on using the crypt() interface for a cryptography project, don’t do it

I understand that “true cryptography” probably requires a different level of cryptographic security than authentication credentials in most cases, but I don’t like to think of it that way. In my opinion, if it’s not good enough for cryptography, then it’s not good enough for the protection of my users’ privacy. Not to mention, the cryptographic security of the password is fixed independently of the size of the password entered. That’s not good.

So what do I recommend?

I recommend encrypting with bcrypt3 4 in your language of choice and a per-row unique salt per password5. Store the encrypted password in a regular varchar column.

Also read Jeff Atwood’s post entitled “You’re Probably Storing Passwords Incorrectly”6.


Footnotes

1chkpass
2crypt
3bcrypt
4bcrypt vs. scrypt
5Where to store password salts
6You’re Probably Storing Passwords Incorrectly

qemu and RISC-V

It’s been a long time since I’ve posted here. So what’s up?

I’ve been working at Lendesk in a team lead / senior developer role and trying to pursue my interest in Go and Linux. This has been a very tricky path to navigate as work has abandoned any pretense of plans to adopt Go and all work machines are MacOS. Thankfully, I’ve been able to carve out some personal time to keep poking around in my area of interest.

I met up with my buddy Lucas for drinks the other day, and as we are wont to do, we discussed Linux, programming, security, privacy and other such things. This got my mind running and eventually I remembered RISC-V1.

A while back I bought a System76 laptop. Originally I had placed an order for a Purism Librem 15, but due to some miscommunication and problems with delivery, I wound up cancelling that order. Luckily, right after I had placed the order, System76 announced that they were removing Intel’s IME from all of their machines. Management-engine-freedom is a great step, but it’s not quite all the way to Libreboot support, or entirely open hardware. RISC-V is entirely open hardware. I’m a huge fan of this.

So, last night I decided that I want to try poking around in the RISC-V world and see what kind of damage I can do. All I managed to do is boot Fedora Linux on a version of qemu built with RISC-V support. The community has done a ton of work here and the process is very easy to get started with; just follow the docs at qemu.org2. I was hoping to start poking around with Go on RISC-V, but doing so requires a cross-compilation build as described on the riscv-go README3.

I will hopefully follow up again with this sometime soon and post appropriate updates here.


Footnotes

1RISC-V
2qemu.org RISC-V
3riscv-go README

Android: Mainframe

After lunch today I made a visit to a local boardgame store just to see what was new. The local gamestore near my place is much cheaper, but has a drastically different selection, so it’s always nice to see what else is available.

There were two notable games on today’s visit:

I’ve seen mention of Agricola: Family Edition before, so that was neat to see in person. I watched a Tom Vasel review of it, and I think I would still prefer to play Agricola: All Creatures Big and Small.

Onto the other game; Android: Mainframe is completely new on my radar. A quick read on the back says:

This sounds absolutely fantastic!

I’ve read two text reviews on Boardgamegeek, and unfortunately this game looks like a dud. I think this is really interesting, because I’ve been seriously thinking about Android: Netrunner a lot lately. It’s a game that I think is really interestingly broken. I won’t go into all the details here; I’ll just say that I’ve written pages of notes on it and am trying to identify what I like in the original design.

I was excited to see this two player strategic boardgame in the Android universe because there seems to be a large crossover with my thinking on Android: Netrunner. Things I would have liked to have seen in this version:

A quick glance makes it obvious that none of this applies to Android: Mainframe. This game is simply a rethemed version of Bauhaus1 with some random elements added via your “powers” in your hand.

Bauhaus looks pretty fun, but I think the attempt to theme it and add random elements probably subverts the basic game. Oh well. I guess this is good motivation to keep doing my thinking and planning on an Android: Netrunner influenced game design of my own! :-)


Footnotes

1Bauhaus boardgame

Race for the Galaxy Android

Race for the Galaxy is out now on Android1.
This features Keldon Jones’ AI2 and is the best digital version I’ve seen. It translates wonderfully and is easily worth the ten bucks.


Footnotes

1Race for the Galaxy on Google Play
2Keldon Jones’ Race for the Galaxy AI

Editor Sizes

A few months ago I was starting to get irked at the relatively poor performance of Emacs on my primary development system. Emacs isn’t terrible, but it is noticeably slow compared to Vim. So, for the sake of curiosity I decided to look at some executable sizes. I started by just doing a simple:

ls -alh `which <editor>`

But that only captures the size of the executable on disk without regard for any shared libraries or dynamic memory allocations.

I can’t recall where, but I found a comment on some forum that suggests the following:

ps -Ao rsz,vsz,cmd | grep <editor>

This works well. rsz and vsz are Resident Size and Virtual Size, respectively. cmd gives the name of the executable. Output looks like this:

$ ps -Ao rsz,vsz,cmd | grep [s]t
8896  65592 /usr/local/bin/st

The resident size is how much memory is actually in use by the application and the virtual size is how much memory the application has requested. There’s usually a rather large delta there, and it’s kind of interesting to see. In both cases the size is given in bytes, so divide by 1024 to get Kibibytes and by 1048576 to get Mebibytes. (Kilobytes and Megabytes in common terminology.) I did an audit of various text editors running in terminal or gui mode and captured their resident memory sizes which I’m sharing below. Pay attention to the suffix because there are many orders of magnitude differences between the smallest and largest programs!

In no particular order:

Editor Name Resident Memory Size GUI or TUI Additional Information
emacs 57.99 MB GUI No local settings files
emacs 22.43 MB TUI With local settings files
emacs 12.77 MB TUI No local settings files
nano 1.77 MB TUI
uemacs/pk 916 KB TUI https://github.com/snaewe/uemacs
jed 1.6 MB TUI
gvim 25.8 MB GUI
vim 25.8 MB TUI
atom 266 MB GUI
nice 1 MB TUI http://ne.di.unimi.it/
micro 10.95 MB TUI https://github.com/zyedidia/micro
vis 3.88 MB TUI https://github.com/martanne/vis
TextAdept 25.58 MB GUI
TextAdept 6.58 MB TUI
jupp 3.37 MB TUI https://www.mirbsd.org/jupp.htm
mg 1.03 KB TUI Micro GNU/emacs, this is a portable version of the mg maintained by the OpenBSD team.

NOTE: TUI means Textual UI and you’ll often see “curses”, “cli”, “console” or “terminal” used to mean the same thing.

Don’t take these records as gospel; I’ve noticed differences on subsequent captures of various programs. Without a doubt each editor will perform differently depending on the specific file and how the editor represents said file in memory. Also, I generally don’t have a good understanding of Linux memory, so there are lots of ways these numbers could be non-representative. As a ballpark estimate, however, I suspect these numbers should suffice.

If you liked this, you might enjoy A Memory Comparison of Light Linux Desktops

Android Applications

I’ve been getting frustrated with my computing experiences lately. I won’t go into many details here, but my frustration has extended to my smartphone and has spurred me to investigate alternatives to many programs and services I’ve been using.

Provided for your benefit here is a list of several Android programs I’ve recently discovered. I’m not vouching for these yet, and may not even have much experience with them yet, but they are here and may serve you well.

Name Description On F-Droid?
Syncthing P2P File Syncing Yes
Ted Text Editor Yes
DAVdroid Caldav and Carddav client Yes
VLC Media Player Yes
Firefox Web Browser Yes
Anuto TD Game (Tower Defense) Yes
Document Viewer Views PDFs and other files Yes
Google Authenticator Two Factor Auth Yes
FreeOTP Two Factor Auth Yes
Hacker’s Keyboard Keyboard Yes
Wallabag Read-It Later/Pocket Alternative Yes
Antennapod Podcast Client Yes
Swiftnotes Notes Yes
Forecastie Weather Forecasting Yes
SkyTube Youtube Yes
Clip Stack Clipboard Manager Yes
OsmAnd Maps Yes
Torch Flashlight Yes
Focal Camera Yes
Vanilla Music Player Yes
Barcode Scanner Barcode Scanner Yes
Mathdroid Calculator Yes
Ghost Commander File Manager Yes
Gallery Photo Viewer Yes
Timer Clock, Stopwatch, Countdown Yes
Night Screen Screen Brightness/Coloration Yes
Bubble Comic Book Reader Yes
Android IMSI-Catcher Detector Yes
Yaairc IRC Yes
KeePassDroid Password Manager Yes
Termux Terminal Emulator Yes
aSQLiteManager SQLite DB Manager Yes
BubbleUPnP DLNA Client No
OWLR Foscam IP Camera Client No
Zarchiver Archive Expander No
Private Internet Access VPN No
Protonmail Email No
Signal Messaging No

HandmadeCon 2016 and Self Hosting

HandmadeCon 2016 is just around the corner!
I’m really excited to go!
The speaker lineup looks absolutely stellar this year.

Besides that, I’ve obviously been keeping myself busy for a few months here with nary an update. The most visible change to anyone who might read this blog is that I’ve completely revamped my hosting situation; both making great progress on my goal of being entirely self-hosted, and for serving up content via HTTPS to you.
The specific implementation by which I’ve accomplished this is pretty interesting and has been a great opportunity to learn. I have documented a large portion of what I’ve done and I hope to share that here.

Short and simple. Talk soon.

Linuxing

Have you ever heard of suckless1? I think I stumbled onto suckless sometime while reading K. Mandla2, though there’s a reasonable chance it might have happened before then. suckless immediately resonated with me because of their philosophy3. Their software also looks good, very accurately reflecting the idea of minimalism4; often requiring you to rethink how you typically interact with software to do what you want to do. Sometimes this process of re-evaluation is fruitful, sometimes maybe less so, but it’s always interesting to experience. They have their own Linux distribution, called stali5 which I really recommend reading about. I’ve found a couple of other distributions that are similar in various ways: CRUX6, Tiny Core Linux7 and Morpheus8. Morpheus in particular looks to be faithful to the suckless way. I think CRUX and Tiny Core Linux are both full GNU stacks, so they’re less interesting to me personally, but if I struggle to be productive with the more experimental stali and Morpheus, then I wouldn’t hesitate to give them a try.

There are multiple facets to minimalism. Morpheus and stali both adopt non-GNU tools like musl9, sbase10 and ubase11. (Though stali doesn’t officially adopt all of these yet.) I like the idea of this, because GNU tools are often huge, complicated pieces of software with all kinds of support, sometimes for legacy systems that are practically ancient. The idea is also novel to me. I had simply never considered using a runtime libc other than glibc before. Clang isn’t new, so the idea of using a non-GNU compiler isn’t quite as novel, but in practice it’s still not common. Have you ever thought about the organization of a Linux distribution? Prior to reading about stali, I hadn’t. Linux distributions ship with dynamic libraries in common locations and programs are built to dynamically load these libraries at runtime. This can cause some problems:

Armed with this new knowledge, I’m really curious to try building statically linked programs with a non-GCC compiler using something other than glibc. Tiny C Compiler (TCC)12 caught my eye, but I had issues doing what I wanted. I can’t recall exactly what I encountered. I also gave Clang a whirl13 but that similarly didn’t want to work for me. I did get something going with musl and GCC14, but that’s clearly just a piece of what I want.

Okay, sorry, minimalism. Another aspect here is system initialization. I guess there’s a conflict in the larger Linux community over systemd. I’ve never really been curious about init systems before and my only prior exposure was invoking systemctl or /etc/init.d/... or service [start|stop|restart] .... When I started exploring Manjaro Linux, I also saw two variants (regardless of the specific desktop environment being included): with systemd or with openrc. I had no idea what that meant at the time, so I didn’t learn anything from it. suckless link to information about systemd on their stali page and I learned a lot from that. I can’t say I actively dislike systemd because it’s in my Arch installation and it appears to be working well enough. However, I’m now curious about alternative init systems and especially in trying to pare things down to their simplest minimal subset. suckless have sinit and all the distributions I linked above use something other than systemd.

So all of these things are very interesting to me and I’ve been downloading images and trying to use alternative Linux systems for just the tiniest amount of time now. It’s a learning experience for sure. Both Morpheus and stali lack less, more and various other tools I’ve come to rely on. We’ll see how things continue.


Footnotes

1suckless.org
2Motho ke motho ka botho
3Suckless Philosophy
4Lessism Over Minimalism
5stali
6CRUX
7Tiny Core Linux
8Morpheus
9musl
10sbase
11ubase
12Tiny C Compiler
13How to compile C apps with musl and Clang
14GCC + musl

Bluetooth on Arch

Well, today I had a “fun” little diversion.
I started out by wanting to connect a game controller to my crappy little Dell Mini so I can start prototyping some games. Of course a controller isn’t required, but I like controllers and it seemed like a small little learning exercise.
If you know me and my experience with computers, then you’ll know that this didn’t go as planned. Also, why would I be writing about it if it did? :-)

I’m running a console-only installation of Arch Linux with systemd. That should give you context for the following, and if your environment is different then you’ll have to modify the appropriate commands accordingly. I’m also talking specifically about using a USB bluetooth adapter, though I just checked and apparently this little netbook has built-in bluetooth.

Before starting, let’s assess our current USB situation:

lsusb

Take note of what’s showing. Now plug in the usb bluetooth adapter and check again:

lsusb

Make sure the device shows up properly.

Now, I don’t have any bluetooth utilities installed by default, so I have to set that up:

sudo pacman -S bluez bluez-utils
modprobe btusb
grep btusb /proc/modules # Make sure the kernel module is loaded
systemctl start bluetooth
systemctl status bluetooth # Verify the bluetooth service is working

At this point I had a problem.
The primary issue is that trying to execute scan on from bluetoothctl would tell me that the bluetooth device wasn’t ready. I had a difficult time resolving it, and started resolving a secondary issue that may or may not have been related. I noticed in the systemctl status bluetooth call an error message regarding sap-server which lead me to some resources.1 By following a comment on that bug, I was able to resolve the problem.

sudo mkdir /etc/systemd/system/bluetooth.service.d
sudo $EDITOR /etc/systemd/system/bluetooth.service.d/customexec.conf
# Now in customexec.conf:
[Service]
ExecStart=
ExecStart=/usr/lib/bluetooth/bluetoothd --noplugin=sap
# Out of the file now:
systemctl stop bluetooth
systemctl start bluetooth
systemctl status bluetooth

Now that problem has been resolved. But I still can’t scan…
Thankfully the solution is extremely simple:

bluetoothctl
> power on # This is the problem!
> scan on

Voila!

Well, the whole goal was to get a controller talking with my laptop which I opted to do via bluetooth and I got bluetooth working… however, I brought a PS3 controller to try with. Unfortunately, PS3 controllers use a proprietary protocol on top of bluetooth and can’t be paired with normally. See more info.2 I should have known this already, because you have to connect a PS3 controller via USB cable to a host PS3 system to get it to pair with that system. Well, luckily enough there is a utility you can download called sixpair that’s available in the AUR. Unfortunately, simply using sixpair on its own isn’t enough.

And thus, the end of my journey with connecting the PS3 controller to my laptop. At least for now.


Footnotes

1Arch Bluetooth Issue
2PS3 Controller Bluetooth Guide

Brief Update for 2016-05-11

I haven’t been too active here or on my daily coding project1. So what’s been going on?
Honestly, not too much. A lot of Dark Souls. I mean watching VaatiVidya2 and EpicNameBro3 while also playing through Dark Souls: Prepare to Die Edition4 and Dark Souls II: Scholar of the First Sin5. I love that series so much. I’m not very good at it and I don’t like everything that the games do or that the community is interested in; but overwhelmingly I am completely entranced by it.

Besides videogaming and family business, I’ve been doing a little bit of software tinkering. However, this tinkering is a lot less on the programming side of things and much moreso on the configuration and setup side of things. I’ll document these things more in the near future. Here’s a quick summary of what I’ve been playing with:

- Mumble Raspberry Pi Server
- SSD Nodes VPS for my domain name and to forward network traffic to my home servers
- X-less Arch installation on my Dell Mini 10 (ie., console-only)

And some things I’ll be playing with soon:

- Much more console-land exploration on Arch
- Sabotage and/or Tiny Core Linux exploration
- Linux From Scratch study
- More electronics (when I get my AliExpress shipments)
- More home servers
- Linux Kernel programming study
- Linix Framebuffer programming study
- Rust language study

And more family time, more workout time and more Dark Souls. :-) I plan to write on all of this stuff and I think having my new setup on the Dell Mini will help achieve this goal.


Footnotes

1BitBucket Daily Coding Practice
2VaatiVidya on Youtube
3EpicNameBro on Youtube
4Dark Souls: Prepare to Die Edition on Steam
5Dark Souls II: Scholar of the First Sin on Steam