September 2009 Archives

As it turns out, Todd actually put together a list after a month long sojourn to Morocco a few years back. I thought this was a hell of an idea, and asked him if I could expand on it. He agreed, and the following list was born. I actually like to think of this as more of a living document, since future travel experiences should add to it’s content, suggestions, and overall relevance.

The end of the line for RAID?

The Beatles remasters have finally hit the street and all across the world, music fans are gorging themselves on the most fabled and revered repertoire in pop music history.

One of this blog's regular correspondents has just been attending a chemistry outsourcing conference (program here), and heard a very interesting talk from Stefan Loren of a Baltimore investment advisory firm, Westwicke Partners. Loren's a product of the Sharpless lab, who went on to Abbott, then Wall Street (Legg Mason and into the hedge fund business), and had some very provocative things to say about our industry:

We've seen a lot of NoSQL action lately built around distributed hash tables. Btrees are getting jealous. Btrees, once the king of the database world, want their throne back. Paul Buchheit surfaced a paper: A practical scalable distributed B-tree by Marcos K. Aguilera and Wojciech Golab, that might help spark a revolution.

In Defense Of Food (Network)

In essence, data stores are distributed key/value stores that provide unlimited scalability to store data that is closely modeled to objects removing the need for ORM plumbing code. The downsides are the loss of data integrity, which has to be managed by application code and the difficulties in performing business intelligence on the data.

Database normalization is a technique for designing relational database schemas that ensures that the data is optimal for ad-hoc querying and that modifications such as deletion or insertion of data does not lead to data inconsistency. Database denormalization is the process of optimizing your database for reads by creating redundant data. A consequence of denormalization is that insertions or deletions could cause data inconsistency if not uniformly applied to all redundant copies of the data within the database.

The difficulty in these systems comes with the fact that large amounts of data need to moved around every day. Thus although hundreds of gigabytes or terrabytes of data are not to difficult when sitting still in a storage system, the problem because much, much harder when it must be transformed to support quick lookups and moved between systems on a daily basis. This post describes the system we built to deploy data to the live site using our key-value storage system, Project Voldemort.

GemStone has unveiled GemFire 6.0 which is the culmination of several years of development and the continuous solving of the hardest data management problems in the world. With this release GemFire touts some of the latest innovative features in data management.

I few weeks ago, I posted about a conversation I had with Jeff Hammerbacher of Cloudera, in which he discussed a Hadoop-based effort at Facebook he previously directed. Subsequently, Ashish Thusoo and Joydeep Sarma of Facebook contacted me to expand upon and in a couple of instances correct what Jeff had said. They also filled me in on Hive, a data-manipulation add-on to Hadoop that they developed and subsequently open-sourced.

I ‘m getting a lot of questions lately about the distributed computing, especially distributed computing model, and MapReduce, such as: What is MapReduce? Can MapReduce fit in all situations? How we can compares it with other technologies such as Grid Computing? And what is the best solution to our situation? So I decide to write about the distributed computing article in two parts. First one about the distributed computing model and what is the difference between them. In the second part I will discuss the reliability, and distributed storage systems.

An interesting observation in the talk is that the more robust products are internal to large companies like Amazon and Google or are commercial. A lot of the open source products aren't yet considered ready for prime-time and Bob encourages developers to join a project and make patches rather than start yet another half finished key-value store clone. From my monitoring of the interwebs this does seem to be happening and existing products are starting to mature.

attempted to pull together a cross platform Nagios plugin that did it's best to give me what I wanted, and what do you know, it works!

HBase: Bigtable-like structured storage for Hadoop HDFS

This article is about how to monitor Linux and Windows hosts with SNMP (2c version) and Cacti.

While playing around with initrd images a few weeks back, I came across the mkinitrd “–with” option. This option allows you to add additional modules to an initrd image, which is useful when you have a new storage or Ethernet driver that isn’t supported by the base operating system.

if you pass a variable to your job through qsub or qrsh with the -v switch, and if that variable starts with SGE_COMPLEX_, the SGE_COMPLEX_ part will be stripped off, and the remainder will be treated as a resource request whose value will be placed in the job's environment.

In this instructable im going to show you how to make a quick, cheap and easy Beneficial Bug Houses!

Using the Amazon EC2 service as a case study, we show that it is possible to map the internal cloud infrastructure, identify where a particular target VM is likely to reside, and then instantiate new VMs until one is placed co-resident with the target. We explore how such placement can then be used to mount cross-VM side-channel attacks to extract information from a target VM on the same machine.

Yesverywellbut, say those comfortably removed from the economic margins, such fees can be avoided through responsible account management. And the supercilious and dim yesbuts raising this objection will go on to point out that they, personally, have never had to pay such a fee. And they will take this as evidence of their own superior responsibility and their oh-so-superior superiority too all of those stupid working class people stuck at the margins for whom one flat tire or one sick child or one unanticipated $10 expense can incur a cascading series of late fees and overdraft charges and other forms of emergency short-term credit that can easily exceed the $150 that Wal-mart graciously offers to extract from their paychecks each year.

It mostly seems like a slightly superior incarnation of The Rolling Stones’ Their Satanic Majesties Request, a record that (ironically) came out seven months after this one. Pop archivists might be intrigued by this strange parallel between the Beatles and the Stones catalogue—it often seems as if every interesting thing The Rolling Stones ever did was directly preceded by something the Beatles had already accomplished, and it almost feels like the Stones completely stopped evolving once the Beatles broke up in 1970. But this, of course, is simply a coincidence. I mean, what kind of bozo would compare the Beatles to The Rolling Stones?

As I see it, the economics profession went astray because economists, as a group, mistook beauty, clad in impressive-looking mathematics, for truth.

British food does have that reputation– and, of course, you can find bad food here, as you can in most other places. (For example, if an American national should find themselves here, for the love of god don’t order a hamburger.) I’m not about to try to defend haggis or black pudding either, but there are certain things the British do wonderfully well. So here is my official short list of: GOOD BRITISH FOOD ITEMS

This How-To provides step-by-step instructions for creating a Universal Wireless Repeater appliance: a device that you can place anywhere and it will wirelessly repeat the strongest signal, onto another wireless network (with or without security). This functionality is also known as Wireless Client Bridge, or Range Expander. Unlike WDS, once you have this appliance setup, it will work with any open network.

The workplace is not what it was five years ago. Neither is the job hunt. The most successful candidates are those who are ready and willing to adapt to a changing landscape. But it doesn’t matter how ready you are for the modern workplace if your resume ’s straight out of 1994.

Even if you can’t hire a fancy designer and are stuck with Microsoft Word, a few tweaks can turn your blasé résumé into an elegant and functional showpiece.

Bit Calculator - Convert between bits/bytes/kilobits/kilobytes/megabits/megabytes/gigabits/gigabytes.

Whether your manager is consistently delivering this information or not, the feedback, written down, is completely different from receiving it verbally. The path to your brain via the written word is dramatically different than for the spoken word. Reading the highs and lows of the past year makes them permanent and makes them real. And then there’s the surprise.