La Nueva Escuela

Autolock idle Linux TTY Consoles

Wed, 02 May 2018 16:55:43 -0500

I usually lock my sessions, either manually or automatically after some idle time. At home that doesn’t make much of difference, but at the office, or in a public place, it’s important.

However, I’ve always had this secret hole in my security: sometimes, because Linux’s graphical interfaces tend to die, I hit Ctrl+Alt+F[1-6] to use the the real TTYs, curiously named virtual consoles, for the same historic reasons for which the V and the I in VIM mean visual interface. Yeah, google it.

But I digress. The virtual consoles are great, etc, etc. But when I’m done with them and return to my graphical environment du jour, I almost never remember to log out from them. And since I usually use them to troubleshoot stuff, in many many cases, I leave them around with the root user fully logged in.

That’s a problem, no need to explain it.

I searched for a solution before and learned about Bash’s TMOUT environment variable, which you can set up to have Bash log out automatically after an idle timeout. But that’s fragile. Bash can only log out if you leave your session at the prompt. If you’re running anything else like Vim, Bash will not log out, regardless of how long you leave that untouched. I know I’ve had Vims in TTYs running for weeks without noticing it.

Someone online had a slightly more effective approach: use finger or who periodically to monitor idle time and kill the sessions that go over some limit. That’s more secure, sure. But it’s also too aggressive, because it may lose data. What if, for instance, you hadn’t saved what you typed in that Vim?

So, today I came up with an alternative, using GNU screen. The idea is simple, make your TTYs start screen immediately after login. When that screen exits, the session will end. And then idle-locking is trivial, screen has a built-in solution. This is what I did:

Add this at the end of to your ~/.bashrc:

if [ "$TERM" = "linux" ] && tty | egrep -q '^/dev/tty[[:digit:]]+$'
then
    exec screen -c ~/.ttyscreenrc
fi

The exec is important. It ensures that screen will replace bash, in the same process and it’s what guarantees that when screen ends the session will end too.

Create a particular screenrc for this. As seen above, I called it ~/.ttyscreenrc, with this content:

startup_message off
idle 180 lockscreen

And change that file to suit your preferences. I put 180 seconds (3 minutes).

And that’s it. Your TTYs will lock and ask you for a password after the configured idle timeout. And you won’t lose any work.

Share it!

SSH Keys - A Lesson for the Future Me

Wed, 06 Sep 2017 11:49:34 -0500

When I ssh from one system to another on my own network, or between virtual machines, I usually don’t pay much attention to these warnings:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

since I know no one is meddling with my network and I know that’s the result of me reinstalling VMs or reusing IPs etc.

But when I see that while trying to log in to a remote VPS, which I know has been running for a long time without changing either name, ip, or SSH host keys, then I DO WORRY. And that happened to me this very morning, when attempting a login from my laptop.

I’m too much of a nobody to be the target of any hacking, but still, I need to know why I’m getting that warning before I can continue and log in. Thinking “nah, that’s probably nothing” is surely what happens right before you’re hacked for real.

My first step was to try to ssh from a different network and that worked without warning, which was a good sign, regarding my VPS, but it meant that either my ~/.ssh/known_hosts had something weird or I was indeed being attacked, however unlikely.

So all I had left was to think hard and try to remember precisely all I had done since the last time I logged in “warning-free”. And I found the real reason in my bash history:

ssh ateijelo@vps1.example.com -o "ProxyCommand proxytunnel -d vps2.example.com:443 ...."

That was it. I tried to get to vps1 through a restrictive http proxy by using a second server (vps2) that had ssh listening on port 443. And usually, when doing that, I paid little attention to what I wrote after my username ateijelo@... in the command line, since that wouldn’t dictate how to get to the server. It would be the ProxyCommand that took care of it.

But I learned today that what you write there does matter. It determines what gets written to ~/.ssh/known_hosts and in this case it ended up associating the host key of vps2 to vps1. The right thing to do is simple: make sure the hostname in the ssh command matches the destination of the proxy tunnel. Like this:

ssh ateijelo@vps2.example.com -o "ProxyCommand proxytunnel -d vps2.example.com:443 ...."

An Effective Fix for the Switch Pro Controller D-Pad Issues

Wed, 12 Jul 2017 03:45:24 -0500

TL;DR: Open the controller and cover the inner half of the metallic contacts below the D-Pad with some adhesive tape. See pictures below.

The Problem

This is the Nintendo Switch’s Pro Controller:

It was launched along the Switch back in March, and from day one, some people have been reporting that the D-Pad registers unwanted Up and Down inputs when pressing Left or Right. [citations]

I finally got one yesterday and it was, indeed, a defective one. The test that many people performed, playing Puyo Puyo Tetris, failed for me as well. While I could score over 200k points in single player Tetris with the Joy Cons, I couldn’t get to 100k when using the Pro Controller’s D-Pad. And that was mostly because I kept getting unintentional fast-drops.

So, after less that 24 hours of getting it, my Pro Controller looked like this:

A Simple & Effective Solution

I looked around and found several brave souls [citations] that opened their controllers and tried to fix them by modifying some parts or padding in some places with cardboard. But I didn’t want to trim or file down any part, and I wasn’t happy with adding padding that could, with time, change in shape or move out of place, forcing me to open the controller again to “re-fix” it.

While I was still understanding the “mechanical problem”, I noticed that, in the PCB, the gap between the metallic contacts of each direction is aligned with the center. The black rubber thingy that closes each button’s circuit needs only touch one tiny part of both contacts to work.

I suspected that unintended touches were rubbing just slightly the inner parts of this metallic contacts:

So I tried reducing the available contact area by covering just a bit beyond the inner half of all contacts with tape:

This is my (not too pretty) implementation with transparent adhesive, which is also visible in the images above:

Results

This solution is simple since we don’t need to modify anything. And it’s mechanically durable, since the tape rests on the PCB and the button is just pressing down on it.

The result is quite good. Unlike some other solutions I still can get Up and Down when pressing Left or Right if I want to, and I think that’s ok. But I don’t get them unintentionally except if I do Left-Right-Left-Right… repeatedly and quickly.

I just played a round of single-player Tetris after this fix and I got to 190k points without one single unwanted fast-drop.

Not too shabby ;)

PS: Thank You

I’m happy to know my controller has this message from Nintendo inside it:

Thank you, Nintendo.

BotW Panoramas

Sat, 24 Jun 2017 15:13:20 -0500

Creating BotW Panoramic Pictures

Back in February, this post was made in Reddit and I found it fascinating. The world of The Legend of Zelda: Breath of the Wild is expansive and gorgeous. And those panoramic views were great.

That post relied on one key idea: when aiming with the Phrenic Bow, the entire bow disappears from the screen, leaving just the cross-hair. However, most of the process was mostly manual. I wanted to fix that and then improve on some other aspects. My two main goals were:

Make it as automatic as possible
Base it on open source tools

The current result is on GitHub. The code depends on Python 2, OpenCV and Hugin.

If you got this far, I’ll use the opportunity to request help with Hugin. I chose it because it was what I found when looking for open-source tools for making panoramas. But my previous experience with it was close to zero, and I’d like people with more experience to take a look at what I did and suggest improvements.

The Climbing Method

There’s an alternative to using the Phrenic Bow method, that I like to call the “climbing” method. It was suggested in this comment to the original post in Reddit. When Link is climbing a surface, you can move the camera to between Link and said surface, and then Link disappears, giving an almost entirely clear screen except for the hearts.

The field of view with this method is bigger, requiring many fewer pictures to assemble a large view. However, everything is further away from the observer, giving a lot less details in the resulting panorama.

The last picture of the gallery at the end of this post is an example of a panorama built with that method.

The rest of this post focuses on the Phrenic Bow method, which is harder, but gives panoramas with more detail. And the code is written for the Phrenic Bow method only.

Instructions

The code is straightforward, you run the Python script and give it the images as arguments. Wait for a while, and you’ll hopefully get a properly composed panoramic picture. But to get there, you need to follow some guidelines.

1. Dependencies

The code was developed on Debian, and it depends on these packages:

python-opencv
hugin-tools

However, making this run in Mac or Windows should not be hard, as these are open-source.

2. Taking the Screenshots

For best results, follow this rules when taking the screenshots. I did all this on a Nintendo Switch. I don’t know if this can be done on the Wii U’s version of the game.

Use the Phrenic Bow. It can be found in … (can’t remember, tell me in the comments below and I’ll add it here)
Use the Pro HUD. This removes the mini-map, the weather forecast, etc., leaving a cleaner image for the panorama.
Make the pictures overlap. The more they overlap, the better the results. But try to find a balance between overlap and total number of pictures. The stitching progress gets quadratically slower as the number of pictures increases. I try to stay below 100 pictures.
Hold the ZR button when taking the screenshot. This is optional, but highly recommended. That brings the three lines of the cross-hair closer to the center.
Zero Arrows. Also recommended. Use an arrow type for which you have no arrows. When you’re out of arrows and hold ZR, the game fades out the cross-hair a bit, improving the result of the OpenCV’s in-paint call that removes it.

The code works best with no arrows and holding ZR,
(but has options for the other scenarios too).

Column-first. This is optional, but I find it gives me better results: grab your screenshots in a column-first fashion. I find it easier to ensure I don’t leave blind spots in the panorama.

Take screenshots in a column-first fashion.

Disable motion-controls. Also optional, but I found it easer to control the placement of each screenshots if I didn’t have to worry about motion controls. They’re great for aiming; not so much for this.

3. Running the code

Transfer the pictures to your computer. Remember to turn off your Switch (hold power down until the power options menu shows up). Only then extract your MicroSD card. If you pull it out while the system is running, you risk corrupting your data. (Side question: is there a better way to bulk-transfer pictures from your Switch?)

Put all the pictures from the panorama you want to create in an empty directory and run:

python2 botw-panorama.py *.jpg

Of course, adjust the path to botw-panorama.py accordingly.

If you don’t want to hold ZR when taking your screenshots, or you have a lot of arrows of every type you don’t want to throw away you can tell that to the program:

python2 botw-panorama.py --arrows=999 --holding-zr=no *.jpg

This will use a different mask to remove the cross-hair and the number of arrows.

4. Manual Tweaks

The output of the code will most certainly not be a rectangular image. You will need to crop it manually. Like this:

Hugin knows how to crop that automatically, but I’ve found that most of the time I want to make different trade-offs to those of its algorightms. So I do it manually.

Sometimes you get a really weird result, but one that it’s just a Move/Center operation away from a good one. For those cases, the tools leaves around the intermediate Hugin projects it creates. Open one of those, make whatever changes you deem necessary, and then run the tool again with --first-step=N to start from a given step instead of from the beginning.

Gallery

Here are some of my own creations. Click for a closer look.

From somewhere in Tabantha (I don't really remember where).
From the Temple of Time, in the Great Plateau.
Near the summit of Satori Mountain.
Lurelin Village at dawn.
Part of the Gerudo Desert from Wasteland Tower.
Tarry Town at night.
Night from a rock near Goron City
A peaceful afternoon by Woodland Stable
Hateno and the Necluda Sea from the Ancient Lab (using the Climbing method).

Well, fire away in the comments. Suggestions for improvements, GitHub pull-requests and flame wars are all welcome alike.

System Heaven

Thu, 09 Mar 2017 21:35:52 -0600

System Heaven

System Heaven is an easy and secure way of running subprocesses. It is more convenient that subprocess, particularly when pipes are involved. And it’s more secure than os.system because it never uses a shell underneath and it’s not vulnerable to shell injections.

The problems with os.system

While os.system is simple and convenient, it has several downsides:

Insecure: As long as the command to execute is fixed, os.system, as its underlying call system, just works. But as soon as some part of it is variable, there’s a risk of injection. Escaping helps, but it’s a headache, and it’ll break sooner or later.
Inconsistent: The behavior of os.system depends of the underlying shell and the operating system it’s running on.
Limited: The standard output and error output of the command can not be captured with os.system. The return value of os.system is usually the exit status of the command, but it again depends of the operating system and the interpreter.

The problem with subprocess

The subprocess module actually does its job rather well. It solves successfully the issues with os.system and even integrates the capabilites of os.spawn and the old Popen. Its main downside is that some things, particularly pipes, can get cumbersome very quickly.

For instance, these shell backquotes:

output=`dmesg | grep hda`

become the following:

from subprocess import Popen, PIPE
p1 = Popen(["dmesg"], stdout=PIPE)
p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
p1.stdout.close()  # Allow p1 to receive a SIGPIPE if p2 exits.
output = p2.communicate()[0]

The way of System Heaven

The core of System Heaven is the fuction sh. The name of the function, and of the library, is intentionally based on that of the common Unix shells. But System Heaven’s sh is not a shell, nor it ever uses one underneath.

The most basic call looks like this:

from systemheaven import sh
sh("wget http://example.com/something.zip")

We can, however, use variables, which add flexibility while avoiding shell injections:

sh("wget $url", url="http://example.com/something.zip")

You may wonder why isn’t that just as vulnerable to injections as os.system. The difference stems from the fact that System Heaven does an intentionally limited parsing of its command line and variables become one single element of the list that is passed to Popen.

Even if a variable has spaces or other characters that would be special to a shell, it’s safe to use it with System Heaven:

dir = "Some Docs ; rm -rf /"
sh("ls -l $dir", dir=dir) # totally safe

That would produce, at worst, a No such file or directory from ls. This behavior is similar to what module sqlite3 does with the question marks to avoid SQL injections.

A variable can become more than one argument if, instead of a string, you pass an iterable. For instance:

sh("sha256sum $files", files=["file1.iso", "file2.iso"])

or better:

from glob import glob
sh("sha256sum $files", files=glob("*.iso"))

Pipelines can be built very easily with |:

sh("ls $dir | grep txt", dir="/home/user")

The longer the pipeline, the better it feels to use sh:

sh("ls dir | grep a | grep b | grep c")

If you want to read the output of a command, put a | at the end:

count = sh("ls | wc -l |").stdout.read()

With this capability, the pipe of the previous section becomes:

output = sh('dmesg | grep hda |').stdout.read()

To send input to a command, put the | in front of the line:

cmd = sh('| grep py$')
for filename in os.listdir():
    cmd.stdin.write(filename.encode('utf8'))

You can put | in both the beginning and the end of the line. But be careful of potential deadlocks.

Some design choices were driven by this goal. In particular, System Heaven avoids any string manipulation, as explained in the next sections.

Quoting & Escaping

There are no quoting mechanisms in System Heaven. Neither ", nor ', nor \ have any special meaning. Anywhere you feel inclined to use quoting or escaping, use a variable.

For instance, if you say:

sh('ls -l "Some Documents"') # may not do what you're thinking

that will become something like:

subprocess.Popen(['ls', '-l', '"Some', 'Documents"'], ...)

and you will be asking ls to list two directories: "Some and Documents". Probably not what you originally meant. A similar thing would happend with ls -l Some\ Documents. That would list two directories, the first one of which happens to end with a backslash.

In any of those cases, say the following instead:

sh('ls -l $dir', dir='Some Documents')

Variables

Variables only work when used as a whole word. That means that a $ sign in the middle or the end of a word will be passed untouched to the subprocess. For instance, the following:

sh('ls -l /media/$user', user="ubuntu") # won't do what it seems

will become:

subprocess.call(['ls', '-l', '/media/$user'])

and the user argument will just go unused.

Similarly there’s no ${var} syntax. You could use it, but then your kwargs would need a "{var}" key. System Heaven’s parsing is intentionally limited and transparent: a variable is a word that starts with $ and the rest of the word is the name of the variable –without restrictions– and will be looked up, untouched, in kwargs.

The right way to do the thing above is:

sh('ls -l $dir', dir="/media/" + user)

String manipulation is the responsibility of the caller.

Special Variables

Some keyword arguments, like _cwd and _env are reserved for tweaking the behavior of sh. Avoid using variables starting with _.

Examples

The following Bash pipeline:
```
count=$(ls | wc -l)
```
would become this System Heaven invocation:
```
count = sh("ls | wc -l |").stdout.read()
```

System Heaven variables can be iterables. This Bash line:

count=$(ls *.txt | wc -l)

becomes

from glob import glob
count = sh("ls $files | wc -l |", files=glob("*.txt")).stdout.read()

You can change the current working directory with _cwd. This Bash line:

cd /home/user
count=$(ls *.txt | wc -l)

becomes:

from glob import glob
count = sh("ls $files | wc -l |",
           files=glob("*.txt"),
           _cwd="/home/user"
        ).stdout.read()

Linux Console & High DPI Displays

Wed, 01 Feb 2017 00:11:34 -0600

I recently installed Debian 8 on a MacBook Pro with a hi-dpi display, or as Apple calls it, a Retina Display. I used a netinst installer, which is very small, but leaves you, right after the installation, with nothing but a Linux text console.

And the font is tiny!

Luckily, there’s a remedy for this. First, squint your eyes and login, and set the font directly with:

setfont Uni3-Terminus32x16

If you want to try other fonts, check under /usr/share/consolefonts.

To make that change permanent, edit /etc/default/console-setup and make the following three variables look like this:

CODESET="Uni3"
FONTFACE="Terminus"
FONTSIZE="32x16"

Now you can enjoy a readable font size and that will get applied on every boot.

Cut & Paste in the Command Line

Mon, 09 Jan 2017 15:57:58 -0600

Super short post. This may not be a surprise if you’re an Emacs user, but in my circles people use Vim and we discovered this just today. You can cut & paste in the command line, like this:

Cut to the end of the line with Ctrl+K and paste at the cursor position with Ctrl+Y. </center> There are more options than just **`Ctrl+K`** and **`Ctrl-Y`**. Browse the documentation of Bash and search for *kill* and *yank* which are the terms used there for cut and paste, and the reason why the keys are **K** and **Y**. I've needed to swap parts of a command line many times in my life and I always found it difficult. I usually combined the GUI selection clipboard with manually deleting a part of the line. I'm glad to have learned this today.

The HooToo Illustration

Sat, 31 Dec 2016 00:18:41 -0600

My last post got more attention from my friends thanks to the illustration of my HooToo than for the fact that I put OpenVPN inside it. Some doubted I had done it myself, so here’s a little more info.

After I managed to get OpenVPN running I was really excited and I wanted to write a post about it, and I wanted to include an illustration of my HooToo. I took several pictures but I didn’t like any, mostly because the router is worn and because my phone camera is not that great.

So, seeing that it was almost all geometric shapes or text, I tried to reproduce one of the pictures in Inkscape. I chose this one:

I imported the image in Inkscape, zoomed-it big and drew lines and curves on top of it, reproducing the shapes I found. I filled it with colors, gradients and some occasional blurring to give it volume. And I added a dark and blurred copy of the basic square shape as shadow. I also left out details that I thought looked bad, like the icons below the Ethernet & USB ports.

I didn’t record the entire process, not having foreseen this post, but here’s an exploded version of the final drawing.

Of course, you can check it for yourself, just get the SVG.

HooToo, OpenWrt, OpenVPN

Fri, 30 Dec 2016 12:09:23 -0600

Here’s a post for my fellow network ninjas. Yesterday, with some assistance from my networking sensei, Ale Ramos, I managed to install and set up OpenVPN on OpenWrt inside my little HooToo HT-TM02. That’ll be mumbo-jumbo for some people, but if you know what that means, you know it’s a big deal.

Boy, was I excited when that worked! So much so that I spent last night drawing the HooToo above in Inkscape and I’m quite proud of the result. Looks good, don’t you think? Between that drawing and the VPN I’m feeling renewed love for my handy travel router. It’s like a Christmas present for myself.

Ok, the recipe

I won’t extend much. My fellow ninjas (Lu, I’m looking at you) can follow concise instructions. What I’m trying to achieve here is to setup a OpenVPN client connection to my server and then have every wireless station of the HooToo send its internet traffic through that VPN, a common thing used here in Cuba to circumvent internet restrictions (both internal and external).

These were the steps:

Install OpenWrt inside your HooToo travel router. Get started here.
Get your HooToo online and ssh to it.
Run opkg update to update the list of packages. In my short experience, this must be done everytime the router restarts, probably to save disk space. For instance, the HT-TM02 has only 4MB in the root partition.
Install openvpn-openssl and luci-app-openvpn
That second package is a OpenVPN app for LuCi (OpenWrt’s web interface). It looks like this:

Using that interface, create a new config for your VPN. There you can set up your remote host, the port, the protocol, and your credentials, that is, CA cert, client cert & client key. In this point, you should be already able to start the VPN and ping your remote end from inside your HooToo.
Next is forwarding and NAT’ing
Go to Network -> Interfaces and create a new interface. I named mine vpn. Choose Unmanaged for the protocol and select tun0 as the covered device. If it’s not there, add it at Custom.
Go to Network -> Firewall and create a new zone that covers the VPN interface you created in the previous step. Enable masquerading and under Inter-Zone Forwarding enable forwarding from your WLAN zone, that is the zone of your wireless stations.

That should do it.

I must confess I had a hard time getting the firewall to work and I ended up opening it too much. You probably don’t want to accept INPUT traffic on your WAN zone (the one that faces the internet), but you need it for the VPN to work. So, either set it to ACCEPT, or leave it ar REJECT and set a specific traffic rule allowing incoming VPN traffic.

Speak your heart in the comments.

The Illustration

The HooToo illustration above has managed to catch the attention of my friends more than having installed OpenVPN in my real HooToo. So, here’s a short follow-up post about it.

Odoo Inheritance

Sat, 17 Dec 2016 07:36:43 -0600

Odoo Inheritance for the uninitiated

Complex technology can look like magic. And when you use it long enough, magic can become second nature, it can become just “the way it works”. But I don’t like magic when it comes to computers. Specially in a system built with Python, whose Zen includes Explicit is better than implicit.

I’m barely starting with Odoo (formerly OpenERP) and one of the bits of magic that struck me most was its inheritance mechanisms. I read the docs, talked to my senior colleagues and browsed the code heavily. And I wanted to document my findings, from the point of view of a beginner, before the magic became “the way it works”.

Three Inheritance Modes

Odoo provides three ways to inherit from and extend existing models. We’ll check them one by one and, to dispel the magic, we’ll see what’s going on behind the scenes.

First, let’s see what’s common to all inheritance modes. To define new models you inherit, in the standard Python sense, from Odoo’s models.Model, like this:

class Polygon(models.Model):
    _name = 'shapes.polygon'

    pos_x = fields.Float(default=0)
    pos_y = fields.Float(default=0)

Models become tables in Odoo’s database. The _name attribute is how your model will be identified across Odoo, and it’s where Odoo derives the table name from. After you install a module including the model above, you’ll see the following in pgAdmin:

Note how pos_x and pos_y appear as columns in a table shapes_polygon and how Odoo created five extra fields automatically.

Mode One: Classical Inheritance

The first way to extend a model in Odoo is similar, in spirit, to prototype inheritance in other programming languages. You want to define a new model, but you want to reuse the fields and methods of an existing model. You achieve that in Odoo like this:

class Rectangle(models.Model):
    _name = 'shapes.rectangle'
    _inherit = 'shapes.polygon'

    width = fields.Float(default=0)
    height = fields.Float(default=0)

The key attribute here is _inherit. Here you specify the existing model that you want to use as prototype for your new model. Your model gets all the fields of its base model, and it can even use, redefine and access the base methods through super().

With this inheritance, Odoo creates a new table in the database, and includes all the columns of the inherited model. It looks like this in pgAdmin:

Mode Two: Extension

In this mode, you don’t create a new model. You want to extend an existing model in place. This makes it possible to not only add fields and methods, but to redefine existing fields and methods. In this way, your changes may have effect on all current users of the model. This is used very frequently by Odoo’s and third parties’ modules.

The syntax is actually quite simple: include an _inherit attribute but omit _name, like this:

class DescriptivePolygon(models.Model):
    _inherit = 'shapes.polygon'

    description = fields.Char()
    pos_x = fields.Float(string="X", default=0)
    pos_y = fields.Float(string="Y", default=0)