Author Archives: matthewurch

Basics of Distributed Source Control – What is A Changeset?

A changeset is how source control systems keep track of changes.  Everything else is built up around these changesets. But what are they?

Quite simply, it’s a difference.  More specifically, a set of differences. What changes have you made since the last time you committed your code?  These differences are saved as text and tracked by the source control tool.

Creating changesets is referred to as a commit

Imagine if you made a change to a document and someone else asks you, “What did you do to it?”  So, you reply, “I added a line that says…” 

That’s similar to a changeset (for a single file – more on this later).  The person you’re talking to takes what they already knew about the document and adds the new information, the change of the new line, and now can picture the new updated document.

This can repeat for many changes.  “I added another line that says …”  And now the person you’re talking to takes the original document, plus the first change, and also plus this new change, and the final version of the document is known.

If you kept track of these changes, you’d be doing the basics of what a source code repository does.  Consider the following timeline:

  1. Original Document
  2. Add paragraph ABC…
  3. Add line … to paragraph ABC
  4. Change introduction from A to B

You can then answer the same question for each point in time and get a different result: “What is the state of the document at change number X?”  Each time, you start at #1 and add up the changes until you get to the desired point in time.

For a Single File?

Earlier in this post, I said that the document scenario was essentially a changeset for a single file.  But a changeset can be a set of changes all at the same time.  Consider the same document editing scenario, but for multiple documents.  “I added a line to doc A, and I deleted a line to doc B”.  Both of those changes can be saved at the same time in a set, a changeset.

This is the first of a planned series on using distributed source control.  More to come in the future.

Graphing Podcast Networks

I love podcasts.  I listen to them every day.

When I’m looking for more podcasts to listen to, one of my favourite resources is the iTunes website.  The reason for this is because iTunes offers a “listeners also subscribed to” section at the bottom of a podcast’s page.  I can instantly find similar podcasts to any given one.

For example, a great podcast is “Radiolab Presents: More Perfect”.

And if we jump to the bottom of the page, we’ll find five popular podcasts that appeal to the same audience…

We need to go Deeper

Picking one of these naturally takes you to its page and at the bottom again you’ll find yet more podcasts.  So what if we keep going down and down?

Going a mere three levels will get us to 155 podcasts.  Many will be duplicates; for example in the image above there’s Embedded *2, Invisibilia *2, Reply All *2, etc.

Visualizing It

What happens if we crawl this tree of podcasts and graph the connections?  Well, going three levels deep on the podcast “The Vergecast”, we can generate a network that looks like this:

And if we go five levels….

Now what’s really fascinating is the natural clustering that becomes visually apparent.  It makes sense that it would happen but these islands of genres and sub-genres are neat to look at.  Here are a few that jumped out at me, professionally highlighted with Paint:

It’s a fun way of looking at it.

Avoiding Email Scams

Being one of the more technologically capable members of my social circle, I am often asked if a specific email is a scam.

Usually it is.

But what if it seems real?  What do you do if that email honestly feels legitimate?  Well, I’ve been thinking about how to simplify my own rules and actions into general “rules of thumb” for others to easily understand and follow.

Here’s what I’ve got:

  • Is the email unexpected?
  • Get the URL / contact info with Google
  • Google the email itself

Let’s break these down.

Is the email unexpected?

Let me explain this with a couple examples.  If you go to a web page and click “I forgot my password,” I imagine the subsequent “password reset” email is fairly unsurprising.  It’s safe to say that this is not a scam.

Alternatively, let’s say you’ve simply been enjoying your day, doing cartwheels and whatnot, and then you get a “password reset” email.  Here, most likely, someone else is is trying to reset your password, or it’s a scam.  The link may take you to a fake (but identical looking) website, asking you to put your old and new passwords.  You’ll then be handing your password over to whoever runs that site.

If an email is spontaneous, if you weren’t expecting it, there’s a very strong chance that it’s a bad email that means you harm.

But is it possible for an unexpected email to be legitimate?

Yes.

That is the point of email after all, to notify you of things.  Sometimes those things require actions on your part.  Perhaps the site has suffered data loss due to a hack, and wants everyone to change their password.  Well, when something like this happens, consider my next rule of thumb.

Get the URL / contact info with Google

It doesn’t have to be Google, of course.  Feel free to use Ask Jeeves or… Bing.  But the spirit of this rule is to only allow the email to serve as a notification; don’t let it help you accomplish the task.  If it supplies a handy link to take you somewhere, don’t use it.  Pretend you deleted the email accidentally, but still want to do what it said.

Maybe that means going to the website manually and finding the option to reset your password.  Or, maybe that means googling for the contact info, calling, and asking about it – do NOT use a phone number supplied in the email.

The point is that by doing this you’re now doing something outside the control of the scammer. The whole mission of the scam is to scare you into clicking their link or calling their number.

Google the email itself

Another tip is to Google parts of the email.  Often this will take you to discussions about the very same email.  Reading through what others have to say can help you to determine if it’s a harmful email and even help you gain some insite as to how others know.

Hopefully this post can help even just one person avoid even just one scam.

Sharing a Hotel’s Single WiFi Connection Between Two Phones

I recently visited a resort on vacation.  Overall, it was pretty good.  Except for a couple things, like when the entertainers really wanted you to participate in something embarrassing.  Dancing on stage is not my idea of a relaxing vacation, Mia!

Anyways, another not-perfect reality of the resort, was the WiFi.  When we arrived, we found out that the WiFi was not free, and not only that, it was fairly expensive.  $12/day.  Per device!

The per-device restriction was maintained with a captive portal of some sort.  You purchased a username and password to log into the network.  Attempting to use it on more than one device would yield a message along the lines of “maximum sessions exceeded”.

The way to get around this, is to turn one of the phones into a router, of sorts.  It’ll be the main phone responsible for connecting to the WiFi, and sharing this connection with another phone.  The network will just see this as a very busy singular phone.

What you want to do, is use Bluetooth tethering.  And it’s pretty simple to set up.  The below instructions are for Android phones.  I don’t know how they differ or if it’s even possible on iPhone.

  1. Go to the Bluetooth settings screen on both phones and pair them to each other.
  2. On the master phone (the one that will share the connection) find the settings screen “Tethering & portable hotspot”.  In this screen:
    1. Enable “Bluetooth tethering”
  3. On the secondary phone (the one that will connect through the other phone) go to the Bluetooth settings screen again and:
    1. Click the gear icon beside the phone – in paired devices.  This is to manage the settings of the connection.
  4. Check the box “Use for: Internet Access”

That’s it!

Cons

This method is not perfect.  I found it a little finicky.  Sometimes the phones refused to connect to each other, the connection was really slow for the secondary phone, and if the phones got too far apart they would not reconnect automatically upon getting close enough again.

But it was much better than paying an extra $12/day.

Maven – Exclude Resources From Jar

A project I was working on recently had some… interesting packages in the source code.  There was a test package, a config package, and a notes package.  By placing these non-source items in the source folder, they were being included in the final jar that maven would build.

The ideal solution here is to move these packages to more appropriate places, such as a test source folder, a relative config folder in the application’s root, and maybe a private wiki for the team, respectively.

However, not even having enough time to do the work I was assigned, I decided I would at least prevent these items from cluttering up the jar, and I would return to move them properly at a later time.

You’ll want to use the maven-jar-plugin, and configure excludes as absolute package paths:

<plugins>
   <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-jar-plugin</artifactId>
      <version>2.3.1</version>
      <configuration>
         <excludes>
            <exclude>test/</exclude>
            <exclude>config/</exclude>
            <exclude>example/package/path/notes</exclude>
         </excludes>
      </configuration>
   </plugin>
</plugins>

Switching Keyboard Layouts in Windows 10

Adding a New Layout

1. In Control Panel

Use the search box and search for “input”.  Choose “Change input methods”change-input-methods

2. Click “Options”

input-methods-options-link

3. Click “Add an input method”

input-methods-add-input-method-link

4. Search for Your Desired Layout, Select It, and Click “Add”

input-methods-search-and-choose-new-one

5. Click Save

Switching Between the Layouts

The new layout isn’t activated automatically if you left the old layout configured as well.  You can switch between the two by clicking the language menu near the clock.

change-input-methods-afterwards

Sudo on Debian

Sudo Not Installed?

The fix – getting sudo to work – is at the bottom.

When you use Debian for the first time, you may have quickly run into this:

Debian_sudo_not_found2

bash: sudo: command not found

This happens if, during the setup process, you provide a root password.  The Debian installation will install sudo if you do not specify the password.

debian-8-install-root-password

Notice, in the last paragraph of the above dialog:

If you leave this empty, the root account will be disabled and the system’s initial user account will be given the power to become root using the “sudo” command.

And, here is a relevant portion from the installation guide for Debian 8 Jessie:

By default you are asked to provide a password for the “root” (administrator) account and information necessary to create one regular user account. If you do not specify a password for the “root” user this account will be disabled but the sudo package will be installed later to enable administrative tasks to be carried out on the new system.

So, if you do specify the password for the root user account, the sudo package will not be installed.

Using sudo

To add sudo, all you need to do is:

1. Open a terminal and change to root
2. Install sudo

apt install sudo

3. Add user to sudo group

usermod -a -G sudo matthew

4. Have the user log out and back in.

That’s it!

Ubuntu – Automatically Connect to VPN on Boot (Without Keyring Prompt)

This post assumes that you have set up a VPN connection on Ubuntu already that you are able to use, and will show you how to make that connection start on boot.

First, the easy step; tell your existing connection to automatically connect to the VPN:

  1. Open up “Network Connections”
  2. Highlight your default network connection
  3. Click edit
  4. Choose the “General” tab
  5. Check the box “Automatically connect to VPN when using this connection” and choose your VPN connection
  6. Save

automatically-connect-checkbox

Stop the Keyring Prompt

At this point, if you restart your machine, you’ll be prompted to enter your keyring:

unlock-login-keyring

To make this go away on boot, navigate to /etc/NetworkManager/system-connections and look for a file named after your VPN connection.  For me, it was “pia-toronto(openvpn)”.  In this file, make two changes:

  1. Under the “[vpn]” section, remove the “password-flags=1” line.
  2. Add a new section, “[vpn-secrets]” to the file with your VPN account password underneath it:
[vpn-secrets]
password=my_vpn_password

And now you should be good to go.

Using Keepass on Multiple Devices

Keepass on multiple devices

Intro

If you don’t use a password manager, you definitely should.  The security benefits far outweigh the risks.

I used to use Lastpass.  I enjoyed the multi-device convenience, but the idea of trusting just one company so completely that always has a copy of my DB made me uneasy.

Using Keepass arguably still has issues because you are trusting an application, and trust is the opposite of security, but since it’s offline, and I take responsibility for the data sharing/syncing myself, I feel it’s at least less likely that all of my passwords will ever be stolen.

But, I still have multiple devices that I need to access my passwords on: desktop, laptops, phone, tablet, etc.  And, there is truly a need to have automatic syncing.  If you leave it to manual, you’ll forget to copy the DB file after changing that password that you’re trying to use and then you have to wait until you’re on the device with the right version of the DB.

How to Sync the DB File

The program you want to look into is Syncthing.  It does what it sounds like and syncs files/folders across devices.  So essentially, you can now have the Keepass DB file automatically sync across your various devices whenever a change is made.

Here’s why I like it:

  • Support each platform I use (Windows, Linux, Android) and more, but I haven’t tried the others.
  • Encrypted
    • The Keepass DB is encrypted, yes, but it’s still better to keep it from being stolen if at all possible.
  • Open Source
    • You don’t want to use closed source applications where the encryption is important unless you truly trust the company to do it properly.  Having others examine that the encryption is done correctly is a huge bonus.
  • Can be used on machines without Admin rights.

Syncthing Model

For this to really work, you don’t have to connect every device to every other device, instead you’ll need a machine that’s always reachable.  The way my setup works, is that my home computer is always on and the router has port forwarding configured so that Syncthing requests can make it there.

Once you have one reachable Syncthing host, the other hosts just need to connect to this one.  This gives you a connection model that can be visualized like this:

syncthing model

So, let’s consider an example and see how the changes will propagate:

  1. Update password DB on my phone.
  2. My phone connects to my Desktop at home, sees that there is a change and syncs the file
  3. Other devices connect to my Desktop, see that there is a change, and sync the file.

I’ve been using this for many months now and it works very well.

Finding the Versions of All Java Jars in a Directory

Tasked with converting a project to Maven, I needed to find out the version of the dependencies, as not all of them had the version in the name.

jars-without-versions-edited

Looking Up a Single Jar

If it’s a single jar you want to look up, it’s quite easy.  Simply take the sha1 hash of the file, head over to the Maven Central Advanced Search, and put the hash in the “SHA-1 Checksum” field.

maven-central-search-results

Looking Up Many Jars

The above becomes very tedious very quick if you have dozens of jars to lookup.  A whole directory of mysterious dependencies.

Maven Central has a search API that I quickly took advantage of to automate the above process in Python.  I created a script that will, for every file in a given path, hash the file and search for its version.

You simply run it as >jar_version.py <path>

jar-versions-found

The Script

Feel free to use the script.

import sys
import urllib2
import json
import os


def sha1_of_file(file_path):
    import hashlib
    with open(file_path, 'rb') as f:
        return hashlib.sha1(f.read()).hexdigest()


def search_maven_central(sha1_hash):
    maven_hash_url = "http://search.maven.org/solrsearch/select?q=1:%22" + sha1_hash + "%22&rows=20&wt=json"
    response = urllib2.urlopen(maven_hash_url)
    return response.read()


def pull_result_version_from_results(maven_central_response):
    results_parsed = json.loads(maven_central_response)
    if results_parsed["response"]["numFound"] == 0:
        return "Not Found"
    else:
        specific_results = results_parsed["response"]["docs"][0]
        return specific_results["g"] + " " + specific_results["a"] + " v" + specific_results["v"]


def process_artifact(artifact_path):
    sha1_hash = sha1_of_file(artifact_path)
    html = search_maven_central(sha1_hash)
    result_value = pull_result_version_from_results(html)
    print artifact_path + " -> " + result_value


def main():
    path = sys.argv[1]
    if os.path.isfile(path):
        process_artifact(path)
    else:
        for root, _, files in os.walk(path):
            for f in files:
                full_path = os.path.join(root, f)
                process_artifact(full_path)


main()