Today we moved from the 5th floor offices of TravisCI to the 5th floor offices of Co-Up. We cried, but not from all the stairs we had to climb. We cried because we didn’t wanna say goodbye :-(

But, as they say, when one door closes, another one opens. Pretty soon our new desk was set up, our coach Urs had assigned us several challenging tasks, and, along with Urs’ colleague Adam, shared a delicious salad for lunch.

Mostly today was spent working on setting up a rake task that would create a seperate database we will use to test the performance of our data viz appliaction. Because we wanted our rake task to: look for a performance test database, delete it if there was one, then/or create a new performance test database - we had to work out how we could access (unix) system commands and which ones would return something we could use to check the presence of the performance test db. This took AGES.

Tomorrow we’re going to look at caching the contents of one of our (small) tables locally to improve performance further.

If the thought of setting up databases provokes this response:

FEAR NOT!!!

Follow these simple steps instead:

(This is what we did to set up the (development) database we’ll be using for our Farmsubsidy project).

1. Map out the basic data model on a piece of paper, including the relationships between each dataset/table (i.e. one to many, must have, zero or more).

2. Normalise (to get rid of repetition) or De-Normalise (to put it back when removing it is actually not totally logical or necessary)

3. Create a migration file. Name this using the Rails convention: date&timestamp_migration.rb. Store this file in db/migrations/ directory of your application.

4. Write the migration in Sequel, creating tables in the order required for any foreign keys between tables to compute.

5. Create a Rakefile. Write the migration methods here.

6. In terminal create the database (if you haven’t already): createdb databaseName

7. Create a Gemfile that includes all required gems (in this case ‘sequel’, ‘rake’, and ‘pg’).

8. In terminal run bundle install to install the required gems localy

9. In terminal run rake db:migrate to run the migration and create all the tables in the database.

Then enjoy wearing your DataBase forehead sticker with pride:

With Matt’s help we refactored our csv parsing to make it run quicker. We included something Matt likes to do, which is printing a dot every 100 rows. As a consequence, when we ran the file, our screens slowly filled with dots. Actually our screens filled wayyyyyyyyyy more slowly than Matt’s. What took us 3,000 seconds took Matt a mere 175 seconds. We put this down to his SSD. We want an upgrade!

Now we have all the Czech Republic payment data saved into our database. Hurá!!!

We found a new friend! After getting accustomed to our beloved irb we found a new very useful command. It’s “psql”. This command leads us to the PostgresSQL Interactive Terminal and is super helpful. The PostgresTerminal shows us all the databases and their contents, and we love it! Sometimes it has its tics. But who hasn’t?

Today we intended to run a script which was supposed to put 278,601 rows into the database. After calculating that one row takes 4 seconds and that 278,602 rows would take us 1,114,404 seconds(i.e. 309 hours) we stopped the program. We had already spent enough time staring at the screen.

Today was a busy day at the Travis CI office. Not only did Sebastian and Gareth make an appearence (they’ve been out of town over summer), but Julia and Carolina (aka RGSoC Team D*) came to work in the office.

We spent the morning looking through our completed Sinatra tests and editing the names of any test descriptions that weren’t very clear. In some cases this required reminding ourselves of what the test actually did. Carla earn’t herself a 15% More Fun forehead sticker by exploring what params[:agent] returned in the test below (it ‘captures’ the relevant part of the User-Agent header, which in this case is ‘World’).

context 'makes captures in user agent pattern available in params[:agent]' do
	let(:app) do
		Sinatra.new do
			user_agent(/Baz (.*)/)
			get('/foo'){'Hello ' + params[:agent].first}
		end
	end

	it "get /foo & HTTP_USER_AGENT = Foo Bar returns correct body" do
		response = get '/foo', {}, {'HTTP_USER_AGENT' => 'Baz World'}
		expect(response.body).to be == "Hello World"
	end
end

She was happy - especially as she remembered what a ‘capture’ in a regular expression was. (Matt explained it last week. It’s about identifying what the important part to identify is, which you indicate by surrounding with brackets).

Work continued into the afternoon.

Then Konstantin returned from his adventures in North America. We were excited to see him and hear about the french-fry sandwiches his colleagues in Pittsburgh fed him.

We also liked his unequivocal praise (naturlich):