Amazon EC2 – Query Request in Java

My day gig has been having me learn Java and using it to connect to third party providers. If anyone’s ever tried to use Amazon’s AWS APIs, you’ll understand my pain. Although there is a lot of documentation on their site, it’s always a little hard to find exactly what you’re looking for. I didn’t want to rely on one of their SDKs because I try to avoid using libraries where I can. So I spent about 2 days converting their Python code to Java (my Java knowledge is minimal).

Here’s the code that finally worked. I did away with the method, instance variables, and imports. This should get you what you need.

TimeZone tz = TimeZone.getTimeZone("UTC");
DateFormat df = new SimpleDateFormat("yyyyMMdd'T'HHmmss");
final String amazonDate = df.format(new Date()) + "Z";
DateFormat ds = new SimpleDateFormat("yyyyMMdd");
String dateStamp = ds.format(new Date());
String canonicalQueryString = String.format("Action=%s&Version=%s", this.action, this.apiVersion);
String canonicalHeaders = String.format("host:%s\nx-amz-date:%s\n",, amazonDate);
String signedHeaders = "host;x-amz-date";
String text = "";
String payloadHash = DigestUtils.sha256Hex(text);
String canonicalRequest = "GET\n/\n" + canonicalQueryString + "\n" + canonicalHeaders + "\n" + signedHeaders + "\n" + payloadHash;
String credentialScope = String.format("%s/%s/ec2/aws4_request", dateStamp, this.region);
String output = "";
try {
    byte[] signingKey = getSignatureKey(this.secretKey, dateStamp, this.region, "ec2");
    String canonicalRequestHash = DigestUtils.sha256Hex(canonicalRequest);
    String stringToSign = String.format("AWS4-HMAC-SHA256\n%s\n%s\n%s", amazonDate, credentialScope, canonicalRequestHash);
    Mac sha256_HMAC = Mac.getInstance("HmacSHA256");
    SecretKeySpec secretkey = new SecretKeySpec(signingKey, "HmacSHA256");
    byte[] hash = sha256_HMAC.doFinal(stringToSign.getBytes());
    String signature = Hex.encodeHexString(hash);
    String authorizationHeader = String.format("AWS4-HMAC-SHA256 Credential=%s/%s, SignedHeaders=host;x-amz-date, Signature=%s", this.accessKey, credentialScope, signature);
    String url = String.format("%s?%s", this.endpoint, canonicalQueryString);
    HttpClient client = new DefaultHttpClient();
    HttpGet get = new HttpGet(url);
    get.setHeader("Content-Type", "application/x-www-form-urlencoded");
    get.setHeader("x-amz-date", amazonDate);
    get.setHeader("Authorization", authorizationHeader);
    HttpResponse response;
    try {
        response = client.execute(get);
        HttpEntity entity = response.getEntity();
        output = EntityUtils.toString(entity);
    catch (IOException e) {
        throw new BridgeError("Unable to make a connection to properly execute the query to Amazon EC2");
} catch (Exception ex) {
    throw new BridgeError("Unable to make a connection to properly execute the query to Amazon EC2");
JSONObject jsonOutput = XML.toJSONObject(output)

Hopefully that helps someone out!

Form_For Deeply Nested Resource

Sorry folks, haven’t posted in a while. Too busy with the day gig and discovering new technologies (looking at you Elixir/Phoenix).

So, one of my side projects had me creating a deeply nested resource. Essentially:

resources :market do
  resources :stands do
    resources :items

It was hard trying to figure out how to be on the stands/show.html.erb and have a form_for @items. Here’s how I got it to work:

def show
  @market = Market.find(params[:market_id])
  @stand = @market.stands.find(params[:id])
<%= form_for, url: market_stand_items_path(@market, @stand) do |f| %>
<% end %>

Hope that helps someone out there!

Git Git Git!!!

Everyone says Git is a revolutionary tool to manager your repository. Almost all modern development teams use Git…so it should be easy, right? For me, it’s been nothing but confusion. Yes, if you’re working alone and have your own project, it’s easy. Just do a git add, git commit, and git push straight to master and you’re perfectly fine. However, if you’re on a team, it gets confusing real fast.

People work on their own branches, push to your branch, merge branches together, merge into master, make schema changes, etc., etc. etc. It can get complicated really fast. I come from a GUI repository system (Eclipse with AccuRev). Everything was simple. If there were conflicts, you’d be given a nice comparison view. Pulling everyone else’s code was a simple button click. There was no concept of branches. You just keep pulling from the master branch and you’re all set. No on else can push to yours.

There are hundreds of git commands. Now, there are GUI programs that’ll do all the git magic for you, but I hesitate to use them because every team I’ve seen simply type the commands in the terminal. Will it make me less of a developer? Will I be able to speak everyone’s lingo? Are those tools crutches? These are questions that linger over me when I consider using a GUI git program. So, I keep on my path and continue using git in terminal. The reason I’m writing this post? I spent hours on one stupid step I didn’t know I should have taken…but makes sense now that I know.

I only work on my own branch of the repo. When I push, I push to my branch. When I pull, I pull to my branch. I’m hardly on master. That’s for my boss to manager and merge code in to. So, the last time I touched the master branch was 5 months ago. But, I ran into a bug and was wondering if it was my code that was causing it. So, I stashed all my code and switched to master. I played around for hours trying to figure out why I wasn’t on the latest commit. I can see in Github that HEAD was different than mine locally. I switched branches, reverted all my changes, did everything I could do…until I finally asked someone for help. The solution: I needed to do a “`git pull origin master“` even though I was already on master. I, for some reason, magically thought that once I switched to the master branch, I’d have all the latest changes. What I needed to do was pull all the code so I can have it locally…

So, just a heads-up to you all. If you want to have the latest code in the repo, your first step is to do a git pull. In hindsight, I feel like an idiot. Asking that question made it feel like I was brought all the way back to day one. Don’t let it happen you to!

Web Developers Need to Know JavaScript

or at the very least, jQuery. The web is a fast moving target and is constantly being pushed forward. Most websites have some type of dynamic component about them: shopping cart, blog feeds, content that magically appear/reappear when you click on a link. What makes all that happen? JavaScript.

Rails developers are notorious for their hatred of JavaScript. That’s how sub-languages like CoffeeScript emerge. With Ruby, we’re very spoiled. Semicolons, curly brackets, and function names are either nonexistent or minimal. A lot of new developers turn to Ruby as their first programming language, so seeing something like JavaScript is terrifying. As much as we all hate it, you will undoubtedly need to know it. And guess what? It’s actually not that bad.

jQuery is an amazing library. Chances are, the app you’re going to be working on is using jQuery in some form. It gets rid of all the clutter that is JavaScript. It has a steeper learning curve than Ruby, at least at first. But once you can get through the brackets and semicolons, it actually starts to make a lot of sense. The structure of the language will begin to seem logical and, dare I say, you will begin to like having all the parenthesis’ and curly brackets. I’ve even gotten to the point where my Rails apps will have the optional parenthesis’ for method parameters. When I pass in an argument? It’s in the parenthesis.

This is going to be controversial, especially coming from a Rails developer, but avoid CoffeeScript at all costs. It does some things well, but white-spaced languages are evil (looking at you too, Haml). Once you play around with jQuery for a few weeks, it’ll come together. You will enjoy being able to bring dynamicism to your application, even if it is just manipulating the DOM. I consider it more of a procedural type of language. It can be a pain trying to cover every single thing that needs to happen when you’re calling a “`click“` event, but once you figure it out, it’s amazing to see the magic happen in the browser.

Moral of the story:

  • Learn JavaScript/jQuery. You’re not a (full stack) web developer if you don’t know it.
  • Avoid CoffeeScript. Dumbing languages down is not the answer. And, do you really want to go through another preprocessor?
  • Stick with it. It’s intimidating at first and takes a little longer to learn, but once you know it, you’ll see the light at the end of the tunnel.

Not Everyone Has to Know How to Code

In the past few days, I’ve had the privilege of teaching others what I’ve learned so far in Ruby on Rails. I’ve given some folks my Craigslist Clone course for free to get a peer review of what people think. Some are good…some were bad. Constructive criticism is fantastic and welcomed. However, after hearing some complaints, I’m now starting to really believe what I’ve heard from many others: some people are not meant to be developers/programmers.

One of the criticisms I received was that I didn’t explain how I pulled up the search bar when I searched for files. Part of being a developer is learning how to learn. Conducting research is an enormous part of development/engineering/programming. Most folks would simply open up Google and conduct a 2 minute search on how to use their chosen text editor. I was literally told that I shouldn’t expect people to be masters of “Google-fu.”

Another one was that I wasn’t going over what a database schema was or how it all relates to the application. Some folks were looking for Computer Science 101, which can’t be covered in a 4 hour course on how to build an application. I think some people have the misconception that they’re going to receive the equivalent of a semester’s worth of a computer science class, which just isn’t the case. Most developers, especially Ruby/Rails developers, taught themselves how to code.

Learning how to program is a long, treacherous uphill battle that can never be won. Some of us just aren’t prepared to go through that journey, nor will we enjoy it. Some of us are. If you’ve decided that you want to teach yourself how to code, be prepared. You will experience frustration, loss of hope, and the will to continue. However, for those of us who are able to push through those experiences, there is a light at the end of the tunnel. The euphoria of getting something to work or finally fixing that bug makes it all worth it. It takes a special person to want to go through that process over and over and over again.

So, you have to ask yourself, are you really sure you want to learn how to code?

REST API is Not Simple…

If you’re working as a professional developer, you’ll more than likely be (at the very least) consuming an API. With everything turning to the web, having an API to let outside applications interact with yours (or vice versa) has become a very hot trend. One concept that is really taking the lead is RESTful APIs. I won’t get into the details, but essentially it provides a URL that can be called to do certain things in an application.

Conceptually, it’s easy. A web app like Twitter has an API that you can you use to post your tweets without ever having to log into Twitter. You just need your credentials (ex: username/password), get a few extra things to send along with your request, and you’re good! Well…not really. In practice, it’s a huge pain in the *ss.

There are a million different ways people implement APIs for their applications, so no one application is the same. Also, a simple username/password may not be enough. Sometimes you’ll have to go to the “developer” page of the application. Sometimes that page requires you to sign up (and if the username you use for the regular application is already taken…well, you’ll need to set up a new one). Sometimes, you’ll need 2..5 different “tokens.” Oh, guess what? Those tokens are only valid for 20 days, so you’ll have to generate a new one by logging in to the application again which kind of defeats the purpose of the API.

So if everyone is implementing an API for their applications, why can it be so hard? I have a theory…one that comes directly from my own experience. It really boils down to one person: the developer. They’re the ones building it, and, you guessed it, they’re the ones documenting it. I hate to admit it, but as a developer, I love building things. What’s the thing I hate most? Documenting them.

I think there in lies the problem with APIs. Not so much the technical capabilities, but the documentation that goes along with it. Unless you’re told what to do with the API, you have no idea how to actually use it. The biggest, most well respected companies in the world have some of the worst documentation for their applications. Even if they do, it’s extremely hard to find. And who’s the one to blame? The developer :).

My recommendation to all the development teams out there: hire a documentation specialist!

Heroku: Missing secret_token and secret_key_base for ‘production’

If you’ve been following good conventions when pushing your Rails application out to Heroku, you’ll know that you’re not supposed to push your secrets.yml file to your repo. As such, when you push your code to Heroku, the file won’t be there either. Heroku will try looking for this file, and when it doesn’t find it, it throws this error message:

“Missing secret_token and secret_key_base for ‘production’ environment, set these values in config/secrets.yml”

After spending hours online figuring out how to fix this, there was only one easy answer. You could rely on a few gems out there (Figaro being the biggest one), but you know my philosophy: be a minimalist when you can.

Don’t worry about the secret_token. I think the newer Rails versions deprecated the secret_token.rb file, so you won’t find that if your Rails app is new. To fix the issue, simply add this one line to your production.rb file:

config.secret_key_base = ENV[“SECRET_KEY_BASE”]

This will automagically create a token and assign it to your secret_key_base. Push to Heroku and you’re all set.

Easy as pie.

Best Cities for Ruby on Rails Jobs

Most of us learning how to program are doing it because we want a job. Now, if you’re looking to be an entrepreneur and are learning to program to start your own startup, I still think you should get a job first. I’ve said it before: there is no better way to learn how to code than by getting paid to learn from senior level developers. How do you do that? By landing a full-time gig.

I’ve been keeping an eye on the market to see what the Ruby on Rails job trends are. If you’re looking to become a junior level developer, your chances of getting a job are exponentially higher in a bigger market. It took me a year to get mine because I’m in the Twin Cities. A friend of a friend spent 3 months learning Rails when he landed his, and I’m absolutely convinced it’s because he’s based out of Los Angeles. After desperately looking for jobs for months before (thankfully) getting my current job, here are the cities that I’ve found to have the most Ruby on Rails positions.

**Disclaimer: this is strictly based on me scouring Indeed, Dice, and StackOverflow Careers and seeing the amount of jobs. I didn’t use any sort of advanced algorithm to come up with my findings.

  • San Fransico – Bay Area
    No surprise here. With a new startup seemingly being created every minute, a lot of them turn to Ruby on Rails. That’s how Twitter and a lot of other now-successful companies got started.
  • Los Angeles
    Another huge city in California. In a place full of celebrities and Hollywood wannabes, I’m surprised at the amount of tech jobs in Los Angeles. You don’t often hear about too many companies whose headquarters are here, but there definitely are a lot of job postings.
  • New York
    I love New York, especially the New York City area. I’ve only been there once, but if I was a billionaire (not millionaire, wouldn’t be able to afford it), I’d live here. In a world of stock traders and financial advisors, I am a little shocked that New York has one of the biggest markets for Ruby on Rails developers. I did a triple-take and thought about uprooting my family to move here when I was looking for a job. Thankfully, I found my current one and have a much lower cost of living.
  • Boston
    Facebook, anyone? Boston, along with most of that northeast region, has a strong tech scene. I think all the wannabe Mark Zuckerberg ivy league students are the culprits. There are a ton of startups trying to be the next #{insert startup here}. A lot of jobs out here, if you can handle the crazy amounts of snow they’ve been getting these past few winters.
  • Denver
    Two of the most highly regarded coding bootcamps reside here: Turing and Galvanize. It’s no secret that Denver has become a hotbed for technology, especially web development. We were also considering moving here as well. It has well balanced seasons, never gets too hot nor too cold, isn’t overpopulated, and has a progressive community. A lot of openings here go unfilled.
  • Washington D.C., North Carolina, Atlanta, Austin
    The rest here are in dire need of Rails developers. They’re at that sweet spot where there is an emerging tech market, but they don’t have the supply.

Which city is best depends on where you’re at in your career. If you’re looking to land a junior developer role, you’ll probably want to move to a bigger market like San Francisco or New York. Those places will have all the senior level talent and are more likely able to afford taking a risk on new learners. If you’ve got a few years under your belt and are at an intermediate level, a smaller market might benefit you more. There’s a ton of demand but not enough people to fill the positions. Someone with 3-4 years of experience will look like gold and you’ll definitely have your pickings on who to join. If you’re a senior level developer to knows how to architect an application, forget all of these locations! Find a job where you can work remote!

There you have it. My opinion on which locations have the most amount of Rails positions. If you’re in a small town where there isn’t a tech scene, moving might be a good option. Being around fellow developers and having a community will definitely give you an advantage as far as your learning and your job opporutnities.

Learning and Using Git

Although most tutorials will have you use Git for version control, they can’t possibly go into as much detail as you’d like. Before last week, my Git knowledge was this: git add, git commit, git push. Github has taken the lead in being the repository (repo) of choice for most people/companies. It’s what we use at my current gig. If you’re just starting out and learning how to program, chances are, you aren’t working with a whole lot of people…if any at all. So you haven’t had the opportunity to use Git to its fullest potential.

I’ll be honest, I felt like a complete idiot when I first started pushing code to production. My worst nightmare: pushing code to the main code base and breaking everything. That or accidentally deleting the entire repo. I’ve heard of many SQL commands that omit just a few words or added some accidentally and taking down entire applications worldwide. I definitely didn’t want to do that. But when my manager asked me if I knew how to create a branch or make sure how to get the latest changes that people have committed, I had no clue. So hopefully this will help anyone who’s reading. Here are a few commands I use often and my general workflow with Git:

  • git checkout -b <name> 
    Example: git checkout -b fix-scrolling-bug
    Before working on any issue/feature/bug, you’ll want to create a branch. Some teams use branching differently, but generally, you’d have a branch for the one specific piece of work you’re doing. This helps keeps things concise and will help collaborators understand what your branch is trying to do. So try to name it so people will know what the code is doing. Or, simply put the issue number on it (more on that in a later post). If you’re just working on minor bugs and enhancements, sometimes it might make more sense to have a generic branch that you can push to that you’ll use all the time. More than likely, you’ll never want to be directly on the master branch.
  • git status
    This let’s you see what your current status is. If you’ve made any modifications to your code and it differs from the repo, you can see what files have changed. You can also see what you’ve added to be committed. So depending on when you run this command, you’ll either see red (modified files that haven’t been added) or green (modified files that have been). I like running this command before I add and after just to make sure I’m committing what I want.
  • git diff or git diff <path> 
    Example: git diff app/controllers/users_controller.rb
    I like using this when I’m not quite exactly sure what I’ve changed, especially if I’ve touched many different files. Almost all the time, I’ll forget what I’ve done, so I’ll want to compare my files to what’s in the repo. If I only want to see the changes for a specific file, I can add the file path to the end of the command. I use this frequently throughout my development. I’ll usually use this right after I do a git pull too (mentioned below).
  • git pull origin master
    This will pull whatever changes have been committed by others in the master branch. It’ll sync up your code so that when you’re ready to push to the repo, you’ll have the most up to date code. This is an important step because anytime you push your code, you want to make sure you have all the changes up to that point. It’ll make merging your code to the master branch a lot easier. Plus, you can take a look at whether or not someone has edited code where you’re doing your work. So if you changed something and then someone else changed that exact same line, you’ll have conflicts that you’ll need to resolve, which basically means you’ll need to either fix more of your code or talk to that other person to see who’s code remains. Important command here.
  • git add
    Basic command. Prepare the files you’ve changed to be committed. Make sure you do this before you do a commit or a push.
  • git commit -m “<your message here>” 
    Example: git commit -m “fixing javascript for scrolling bug”

    You’ll almost always want to add a message to your commit. It’s normally just what you’ve fixed. Don’t be too descriptive, just make it a one-sentence line. Chances are, it’s tied to an issue/ticket tracking number (ex: JIRA). Doesn’t hurt to append that number at the end.
  • git push
    Everyone should know this command. This actually pushes your code changes to the repo. If you’re on a specific branch, you’ll be pushing this to that one branch. This is another reason why it’s a good thing to be on a branch versus the master. Even if you push code here that breaks, you won’t be affecting the main code base. Someone will be able to review your code before they merge it in with master.
  • git stash or git stash save “<name>” 
    Example: git stash save “work for scrolling bug”

    This is another command that I highly recommend you use. As a developer, it’s common you’ll be asked to switch gears at the drop of a pin. Chances are, you’ll be right in the middle of something. You’re not quite done with it and it’s too incomplete to commit. That’s when stashing becomes the best option. Running this command will take all of your changes and put it in a stash. It’ll revert all your code back to what it was before. Whenever I stash, I normally like to run a pull too so I have the latest changes before stashing. Or, I want to test original functionality. So I’ll stash my code, test how the app originally worked, and the reapply my changes to test how my code makes it work now. This is a great command.
  • git log
    This is one that I don’t use too often. This is probably more for the person who has to manager the repo, but nonetheless, it can become handy. It’ll give you an audit of all the changes to the repo. You can see who committed code in what branches, when. It’s good when you need to do some detective work to see when changes were made.


There are a ton more commands and even more options you can append to them. Personally, these the most common ones that I use. I haven’t done a whole lot of merging since that’s normally done by the person managing the repo, but I think these will give you a pretty good head start. I know I wish I knew these before going in to a development shop. There are also plenty of free tools to make this process a little easier, like giving you a gui so you can visually see what’s going on (ex: SourceTree by Atlassian). Most hardcore developers will just work straight through the terminal, but if you’re coming from an Eclipse or RubyMine world, having a gui like SourceTree might ease the transition.

Just a final note: if you want to keep your code private, use BitBucket. It’s just another git-based repo. You can have unlimited repos for free and not display your code to the entire world.

Happy gitting!

My Development Philosophy – Be a Minimalist

Having been at my new gig for nearly 2 months now, I’ve learned very quickly what it’s like to use other people’s code. And when I say using other people’s code, I mean using libraries (gems). I’ve always been a proponent of staying as close to the bone as possible. When you start mixing in gems that seemingly make things easier, you begin to rely on them. A great example of this is using the simple_form gem. I have no doubt in my mind that the contributors of this gem know their sh*t and this is not a bash on the gem itself. But what it essentially does is makes Rails forms look cleaner (depending on who you ask). At first glance, things seem perfectly fine. Forms are cut in half because you don’t need a separate line defining a label. But I remember when Bootstrap 3 initially came out and simple_form hadn’t had the chance to update yet. Boy was it a mess. You couldn’t simply define a class on a form like you regularly do because on the backend, simple_form was mutating the default functionality. Long story short, you had to either hack your way around it or wait for the gem to update.

When you start relying on other people’s code, things are fine at that moment. However, as technologies change, you become a lot more susceptible to bugs and upgrade problems. Imagine a Rails 3 application with 50 gems. Now you want to upgrade to Rails 4. Well, guess what? You need to ensure that every gem is compatible, and if not, figure out which ones need to be upgraded or completely abandoned all together. Or let’s say you rely on a particular gem for connecting to your APIs. Whenever an API changes, you have to worry about whether or not it breaks your program. Granted, this is just the nature of software. Things change. But, personally, I want to minimize the impact of change. And one way to do that is to make sure your code is as self sustaining as possible.

The point I’m trying to make is that there are no shortcuts to becoming a great developer. Don’t rely heavily on other people’s code and simplify only when needed. Personally, I’m not a huge fan of compilers that make code look good in the editor but completely different when debugging *cough* Haml CoffeeScript *cough*. I can see using those technologies if you understand what they’re doing. However, I wouldn’t advise a beginner to start outright using Haml or CoffeeScript. Take the time to learn the grass roots of programming languages. Don’t add another layer of unnecessary complexity, or else you’ll always be wondering if it’s your code or the compiler’s/library’s/gem’s. I’m not saying you should learn C or Assembly, but at least learn (for example) vanilla JavaScript. It’s an extremely intimidating language to a beginner. However, if you can climb the initial hump and learn its syntax, you’ll appreciate its complexity. Same goes for any other language.

Be a minimalist. Not in the sense that you use less code to do what you want to do, but in the sense that you rely more on the core language. In the long run, you will become a stronger developer (at least in my opinion). Put it this way: if I were put in a position to hire someone who only knows CoffeeScript versus someone who knows JavaScript, I’d choose the JavaScript person.