Writing Tech Blogs are Hard Work

I once read an article that said more people need to write technical blogs. That the problem with much of the technology field was that people did not write in-depth articles on the stuff that they are doing. And, if more people were able to take the time and post a blog entry here and there we would all be better for it. As nice as that sounds, I have to say that writing technical posts are difficult, time consuming, and can quickly go out of date.

First off, writing in general is not easy, and that is before you get to adding the technical part on top of it. There are a number of brilliant developers, system engineers, and devops people that can create some of the most complex and unique solutions to problems, and yet cannot begin to write the first bit of documentation or narratives to describe what it is that they have done. It is not that they are dumb, they just have not honed the writing skill, or just might not be proficient at it. Writing is like coding, if you don’t use it, you can lose it. Also, it is a skill that can be developed and honed overtime. Myself, my writing skills are rusty after taking a long time off from it, and I am trying to get back into the flow.

There are some people that recommend to become talented at writing, you need to be able to dedicate an hour a day to it, or 5 hours a week. And that is just the writing part. That does not tie in working on the technical elements that are needed to provide content for the audience you are trying to reach. Maybe a few years back when I did not have family obligations this would have been possible, but now I have to sneak in time here and there. And, for the casual tech blogger, this is not going to be the case. Unless you are doing a lot of writing for work, there is little time to develop your writing skills. This leads many people to turn away from writing a technical article even though they might have some of the best ideas, if only they could get them into a usable format.

Now, the next big item on why it is so hard to write technical articles is that gathering together the technical information is not easy. Don’t get me wrong, that is not to say that there are not a number of topic that you can write on. That is far from the truth. There are probably thousands of areas and topics that can be written about. But, just like writing, it takes time to gather that information. Now, there are a few high profile bloggers that are able to dedicate their jobs to writing technical blogs. Many of them are evangelist or full time employees whose job it is to talk about certain technical areas. That is great for them, and to honest, I am a bit envious. For the average person devops engineer or develop it is not so easy.

There are some companies that will allow you to blog on the work you are doing, but for most people that is not the case. So, on top of your full time job, you then most go and use your own resources and your own time to work on getting together the data, the code, and whatever else, to get together the technical information just to begin writing the article. Once the work has been done to vet the project or the topic that you are looking at writing about, you have to circle back around and figure out how it is that you want to get the information together to put it in a story for others to read. This goes back to item number one, and I have already mentioned how difficult even that first step is. Now we are taking it to a higher level by saying you do not just have to write a story, but that you must structure it around the technical information. 

So, after getting the data together and know what you are going to write about, you have to structure it. There are screen shots that have to be made, code snippets that must be shared, links to technical information that must be included. As easy as all of that sounds, it is much harder than it sounds. Figuring out how to clip an image or not make it be 4 megs in size, or how to get the images aligned so that it all looks correct. Then there is the question of where do you put your code you are going to share. How do you get the numbers showing on code? How do you enable syntax highlighting on the code samples? All of that is not easy. All of that can be overwhelming when you just want to write an article to share some information with other techies.

And the last fact of the matter is that you never know if all your efforts are for naught. You could write a great article, but if people don’t find out about it, then what do you do? Do you keep plugging away and hope people will stumble upon the articles you have written? Nobody wants to do a bunch of work for nothing. So, in recap, writing technical articles is difficult. It can be hard, and it is not always rewarding. I thank all the people that stick with it, but I understand all of those who don’t.

Configuring AWSCLI and Python on Windows

Trying and doing stuff on a Windows 10 machine has become a rather interesting experiment. It started as a place to be able to play video games and have access to a few programs that are not easily available on Linux, to a test of seeing if I could now do all the things on Windows 10 that I could do on Linux.

Turns out, I wanted to test out the Cloud Directory service from AWS. I figured the 2 easiest ways to do this would be via the AWS CLI and Python. It did not occur to me that I had neither of these installed until I opened up Cmder.exe and typed

aws
and then
python
and both came back with the not found.

Wait, what? Where are my programs!

So, now I need to install and configure both of these. The test is to see how easy or difficult it is to get this setup on this Windows machine. Quick list of the normal steps that I take to install the AWS CLI on most any Linux machine.

  1. Install Python
  2. Configure a virtual environment to hold my cli tools
  3. Use pip to install aws cli
  4. Configure aws cli
  5. Test that it all works

The first step in setting all of this up is to get Python installed on your machine. The AWS cli is based on Python, and as such you need to have python installed in order to use it. Now, there are some that will install the AWS cli to the root of the machine, and use the system’s globally installed Python. Due to having worked on multiple versions of Python at the same time, and projects that use different libraries, I almost always setup an Virtual Environment to run my Python programs and other sundry programs from. This way, I don’t cross contaminate my streams, and have a clearly defined idea about which versions I am using on different projects.

Installing Python

This is a relatively straightforward task. Click on the Python installer that suits your needs, download it, and follow the install prompts. I chose Python 3.6.7 because it is the version that I am already using when running some Lambda programs in AWS, and because there are some new changes in 3.7 that have broken a few other libraries. On big one is ‘async’ and ‘await’ now becoming keywords. Follow the prompts to install Python and the restart your favorite command line tool. I run bash via git, and use cmder.exe as my shell program.

Once you have it installed you should be able to run the following to verify that you have install python on your workspace. python --version This should output ‘Python 3.6.7’ or whichever version of  Python that you installed.

Setting up Virtual Environment and AWS CLI

The next part is to install the virutal environment and to then use that to install the aws cli. This should be able to be done with just a few commands, and then you should be up and running.  First we run python and install the virtual environment. Then we activate the environment and install the AWS cli. It is just a few simple commands, and you should then be up and running.

c:\ericv\dev\python -m venv p367
c:\ericv\dev\> p367\Scripts\activate.bat
(p367) c:\ericv\dev\> pip install awscli
(p367) c:\ericv\dev\> aws help

And bamm! you are done. Now, there is always configuring aws to use params, but that is another issue. But, it took me longer to write this up, than it took me to do the install.  That in of itself is a good thing to know. Now, the question is if I will run into any more problems. But, so far so good.


Issues with Ubuntu’s Startup Disk Creator for non-debian ISOs

Let me start with the scenario that led me into issues with the Ubuntu Startup Disk Creator. I had been running Ubuntu GNOME, a flavor of Ubuntu that was focused on a mostly vanilla install of Gnome on top of Ubuntu. Well, Ubuntu was finally getting rid of the Unity desktop, so the spin off that I had been using was no longer going to be updated. Fair enough.

I had Ubuntu installed, and I wanted to give Fedora a try (it has been a while), so I just needed to create a bootable USB stick. Normally I would use dd, but Ubuntu has a tool, and I thought sure, let’s try this gui tool. Quick and easy. That led me down the rabbit hole that you see here.

Ubuntu has a page dedicated to the topic. Create a bootable USB stick on Ubuntu. This page walks through using the Startup Disk Creator. It does mention using an Ubuntu ISO image, but what should work for one, should work for most any ISO. At least this is what I thought. It turns out that if you are not using a Debian based distro, then the application will fail silently. It just sits there and does nothing.

At this point, I could have just used dd and been done with it, but I wanted to find out what was going on with the application, and why it was not working. Let me add a quick note and say, I had not tried a Debian based ISO on the application. This was because I wanted to try out the latest Fedora, and had not bothered to pull down another ISO.

Finding the source code for the USB Creator took a bit longer than planned. The code is hosted on LaunchPad which uses Bazaar as its version control system. Having used multiple systems over the years, launchpad felt like a step back in time. Unless I missed something, there is no easy search within a project, the navigation is antiquated, and the look and feels leaves a bit to be desired.

My first thought was to create a bug on the issue. Even if I was going to fix it, I wanted to ensure that the issue was being tracked, and to see if anyone else had submitted a bug on the topic already. I was surprised when I went to the bug page, and found that the package had not been configured for bug reports yet. At this point, I was more frustrated than anything else, and had decided I was going to use dd to create the usb disk, but I wanted find out what the problem in the code was.

Digging through the code, I found that the core of the application is a few python scripts. Nothing wrong there. I am a huge fan of python for a number of reasons. So, I dug into the code and found the issue rather quickly to my surprise.

From https://bazaar.launchpad.net/~usb-creator-hackers/usb-creator/trunk/view/head:/usbcreator/backends/base/backend.py

        if extension == '.iso':
            label = self._is_casper_cd(filename)
            if label:
                self.sources[filename] = {
                    'device' : filename,
                    'size' : os.path.getsize(filename),
                    'label' : label,
                    'type' : misc.SOURCE_ISO,
                }
                if misc.callable(self.source_added_cb):
                    self.source_added_cb(filename)
        elif extension == '.img':
            self.sources[filename] = {
                'device' : filename,
                'size' : os.path.getsize(filename),
                'label' : '',
                'type' : misc.SOURCE_IMG,
            }
            if misc.callable(self.source_added_cb):
                self.source_added_cb(filename)

The issue is on line 42. The application checks to see if the iso file ‘is_casper_cd.’ This check returns `None` if it does not follow this format. A simple exception could have given the end user some sort of idea about what the issue was, but instead, it fails silently.

From https://bazaar.launchpad.net/~usb-creator-hackers/usb-creator/trunk/view/head:/usbcreator/backends/udisks/backend.py

    # Device manipulation functions.
    def _is_casper_cd(self, filename):
        for search in ['/.disk/info', '/.disk/mini-info']:
            cmd = ['isoinfo', '-J', '-i', filename, '-x', search]
            try:
                output = misc.popen(cmd, stderr=None)
                if output:
                    return output
            except misc.USBCreatorProcessException:
                # TODO evand 2009-07-26: Error dialog.
                logging.error('Could not extract .disk/info.')
        return None

The fact that this code fails silently, the code base does not provide for bug features, and that there is no documentation on this bug/feature is problematic. A simple review of this should have caught the problem. Also, as you can see in the previous section, they do log an error when the data can not be extracted, but they do nothing if the file format does not exist.

At a minimum they could have thrown an error. Another option would have been to continue with the disk write, but to skip the label information. Otherwise, give the end user a clue.

In the end I just used good old fashioned `dd`.

user@host$ sudo dd bs=4M if=/path/to/archlinux.iso of=/dev/sdx status=progress oflag=sync