• All Content
• Articles
• Links
• Books
• Comments
• Tags
• Contact
• OmniFlow

Most Nerd-Its | Nerd Trends | Last Ten

What is OmniNerd?

Voting Booth

Background

I remember when I first discovered MP3 files back in the early 1990s and tried to tell my fellow college students all about how important they were to the future of the world. You could fit a whole album onto a Zip Disk!¹ While this was a huge breakthrough in my mind, one friend kindly pointed out that Zip Disks were significantly more expensive than store-bought CDs; it was highly unlikely that the Music Industry would embrace them as a distribution media.²

Of course, now everyone is using digital music. This technology has brought with it not only the ability to have your entire library of music stored in one place, but also an abundance of music manipulating and categorizing features, most of which are available via ‘tags.’³ Unfortunately, my experience is that most of these features are neglected, something especially true in iTunes. Most kids’ iPods I see have improperly tagged music, to the extent that even the artist names are a hodge-podge of permutations (e.g. R.E.M., REM, rem, r.e.m.⁴).

Of course, then there are the other ones – those true heroes of the digital age who ensure that not only does every track have a proper artist, title and album, but also cannot live with the idea of "of" and "Of" co-existing in their library. These pedants rightly demand all tracks also have correct year data, a must if you expect to be able to have a Smart Playlist that only plays songs released when you were in Grade 5 at school. Metadata, a nerdier name for tags,⁵ is your friend; you should learn to love him.⁶ Naturally, I strive to join this elite legion of demi-gods. My next task is to rate every track in my 7525 track library.⁷

Rating iTunes Tracks

Given my obsession with iTunes,⁸ and that I was repeating tasks several times, I began to experiment with the AppleScript interface. PC users may not know, but virtually every Macintosh application has the ability to be automated using a very simple scripting language. While experimenting, I discovered that iTunes stores the ratings not as 0 through 5, but rather as multiples of 20. That is, no-star is 0, one-star is 20, and five-star is 100. It is possible to get, and set, the rating of a track using simple AppleScript:⁹

tell application "iTunes"
    get rating of current track
end tell
p. 

and:

tell application "iTunes"
    set rating of current track to 60
end tell
p. 

Of course, this is not all that useful, as you can easily see/modify a track's rating in iTunes' Column view or Info window.  But, since increasing a rating by one star increases it's value by @20@, it should be possible to increase and decrease a rating by less than this.  This turns out to be true, and ratings that are set for any value, even unlucky @13@, will be stored by iTunes as that value.  

The scope of 101 distinct ratings is somewhat dizzying, but I find six to be far too few and consistently crave at least half stars.  So, I wrote an AppleScript Studio application that displays the current (and previous) iTunes track(s), and allows you to rate them on a @0-100@ scale.  I imaginatively called it iTunesRater.[10]  


    
    

      

        

        
        
        

          
        
        

        
        
      
      

        

          
        
        

          
        
        

          
        
      
      

        

        
        
        

          '''Figure 1.''' iTunesRater, showing my rather eclectic taste in music.|Border
        
        

        
        
      
    
    

Conveniently, with version 6.0.2 of iTunes, we gained the ability to see half-star ratings.  However, there are a couple of "oddities" with the way half-star ratings are handled.  First, only those with an exact half-star rating will display as such, i.e. @50@ will display as two-and-a-half stars while @51@ will display as two stars.  Second, half-star ratings are display only; there is inexplicably no known method of setting such a rating from within iTunes.[11]


    
    

      

        

        
        
        

          
        
        

        
        
      
      

        

          
        
        

          
        
        

          
        
      
      

        

        
        
        

          '''Figure 2.''' Half Star Ratings in iTunes 6.0.2.|Border
        
        

        
        
      
    
    

Since iTunes stores these _actual_ ratings for every track, rather than just rounding them down to the full number of stars, it would imply that when the option _Play higher rated songs more often_ is checked in *Party Shuffle*,  my @99@-rated songs would play more often than those I'd rated @81@.  When I read _How Much Does iTunes Like My Five-Star Songs?_[12] I realised that there was a fairly easy method to check this out.  Thus, _my quest began_.

The Experiment

To avoid tainting my pristene iTunes library in the name of science, I created a new user account. As song ratings (along with playlists) are not stored with the song file,¹³ any ratings changed using this new account would not transfer over to my music library, allowing me to effectively ‘sandbox’ my experiment. Naturally, I backup my iTunes Library File, but one can never be too careful.

Procedure

To automate track creation, I recorded a short (about one second) track, and imported this into iTunes. I then adjusted the track end time to 0.01 seconds,¹⁴ allowing the test to run 100 times faster than having a full one-second track.

I created the following AppleScript to duplicate the file 101 times, rename each track, and give it the required rating:

tell application "iTunes"
	set i to 0
	set base to first item of selection
	repeat 101 times
		set trk to first item of (convert base)
		set name of trk to i
		set rating of trk to i
		set i to i + 1
	end repeat
end tell
p. 

In my case, running the script caused each track to be played once.  I wanted nice clean data to start with, so I wrote another script to set all of the play counts to zero.  This can also be used later on before re-running the same experiment, or to alter the parameters and see what happens.[15]  Select all of the tracks you want to clear the play count of, and then run the script.

tell application "iTunes"
	repeat with trk in selection
		set played count of trk to 0
	end repeat
end tell
p. 

Now, we are ready to run the experiment.  I switched to Party Shuffle, and made sure that _Play higher rated songs more often_ was checked.  Then, I pressed play and went away.

Initial Results

Some twelve hours later, I decided to check in on my experiment. As expected, the zero-rated song was played least (700 times), and the 100-rated song played most (5000 times). However, surprisingly¹⁶ there wasn’t a linear relationship between the ratings. In fact, the frequency jumped up to around 1400 plays for 1-19, then 2100 for 20-39, 2900 for 40-59, 3600 for 60-79, and 4300 for 80-99.

I saved a copy of this data, using the following AppleScript:¹⁷

tell application "iTunes"
	set theData to {}
	repeat with trk in selection
		set theRating to name of trk
		set theCount to played count of trk
		set thisData to {theRating, theCount}
		copy thisData to the end of theData
	end repeat
end tell

get theData

tell application "TextEdit"
	set theDoc to make new document
	repeat with thisData in theData
		set text of theDoc to text of theDoc
                    & (first item of thisData) & ","
                    & (second item of thisData) & return
	end repeat
end tell
p. 

Although I could identify some patterns in the results, I thought a graph would be the best way to interpret the large amount of data collected.  The plot revealed a fairly clear pattern (and not the one I was hoping for), but I decided to leave the experiment running for a bit longer to see if any variation would resolve.  As it turned out, the next time I remembered I had set the experiment up was almost exactly 12 hours later.

Figures 3 and 4 show the relative frequency of each track occuring.  There are clear plateaux for each 'star' rating.  In my first examination of the data, I  missed that this exists for each full-star rating, with the exception of no stars.  That is, the play count for the track with rating 1 is significantly different to that for @0@, whereas the track rated @21@ is the same[18] as the track rated @20@.


    
    

      

        

        
        
        

          
        
        

        
        
      
      

        

          
        
        

          
        
        

          
        
      
      

        

        
        
        

          '''Figure 3.'''  Play Counts after 12 hours.|Border
        
        

        
        
      
    
    


    
    

      

        

        
        
        

          
        
        

        
        
      
      

        

          
        
        

          
        
        

          
        
      
      

        

        
        
        

          '''Figure 4.'''  Play Counts after 24 hours.|Border
        
        

        
        
      
    
    

Repeat of Experiment With a Subset of Data

I re-ran the experiment, this time only using tracks with full- and half-star ratings.¹⁹ It was at this point I noticed the one-half star rated track play count (3973) is different than the no-star rated track (2076). In fact, if the data is plotted, the value we would expect to be the play count for the no-star rated track is actually what we see for the one-half star rated track (see Figure 5).²⁰

Discussion

So, what does this tell us about iTunes and the way it handles ratings? As a hobbyist coder,²¹ there is one big surprise for me. I would have thought that the easiest way to code for the ‘bias’ toward higher rated songs would be to just use the numeric value that is stored as the rating. Clearly, this is not the case, as a 99-rated song plays about as many times as an 80-rated track. Therefore, the algorithm must convert the rating to a number of stars.

The simplest method to do this conversion would be something like: int(rating/20) in python.²² However, as int(19/20) gives 0, a special case coded in for rating==0 must have been used to result in the linear relationship observed between played_count and int(rating/20).

So, is there any point in rating tracks using half-star ratings, or an even finer gradient? Only for tracks rated 1-19; ratings of any other value are no different to rounded down to the nearest star. Ratings that are multiples of ten will result in a purely visual, not functional, indication of preference (so I’ll know at a glance that I like The Songs They Played As I Drove Away²³ just a little bit more than I like If I Could Start Today Again,²⁴ even if iTunes doesn’t.)

¹ Iomega Zip Disks are a high-capacity (100 or 250MB) replacement for floppy disks and were a format of choice for medium-large amounts of data transfer or backup for some time before CD burners became commonplace.

² During the 1990s, PC CD recording technology was virtually unobtainable and broadband, at least where I lived, was unheard of for home users. Zip Drives were the only high capacity removable device available other than sequential tape drives and expensive SyQuest devices, neither of which were suitable for anything other than archival purposes.

³ The most commonly known and used tagging system is ID3, originally an extension to MP3 files but now virtually universally used on compressed music files.

⁴ It seems that no kids today listen to R.E.M., except for that horrible Shiny Happy People song. Yuck.

⁵ Technically, tags are just a subset of metadata. Metadata means ‘data about data’ and includes everything from MP3 tags to the Author information that Microsoft Word sneaks into every document you write.

⁶ Hanscom, Michael. Metadata is a good thing. Michaelhanscom.com, October 24, 2003. Accessed on February 6, 2006. An excellent argument for having accurate and complete metadata.

⁷ A simple Smart Playlist (Rating is 0 stars) can be used to get all unrated tracks. As I write, I’m down to 1244 items in this playlist, but only 1117 are real songs, with the remainder AudioBooks and Podcasts. Naturally, I keep all of the Podcasts and eventually I may even rate them all – just in case I need to listen to my favourite SciPod: The Podcast of NewScientist magazine episode again, you understand.

⁸ Did I mention I was obsessed with iTunes? I wrote a python script to get the required information out of the available data file, which is much faster than querying iTunes for each track. The script gets the play count of each track, multiplies it by the track time and outputs the data in the same format that iTunes shows at the bottom of its window. At the moment, my library usage is approximately: 13896 songs, 42:03:25:12 total time. It’s almost like I’ve listened to every song twice!

⁹ These two scripts will get or set the rating of the track that is currently playing. You can also cycle through a list of tracks, and set the ratings.

¹⁰ Schinckel, Matthew H. iTunes Rater 0.7. Schinckel.net, June 11, 2005. Accessed on February 6, 2006. Even though iTunesRater is still of beta quality, I use it all of the time. I even once had an IM chat with someone else who uses it, so it must be good! Source code is available on request.

¹¹ Since most of my tracks are not rated as exact half-star ratings, but by chance some were, I only came across the display of half-stars by accident.

¹² Hansen, Brian E. How Much Does iTunes Like My Five-Star Songs?. OmniNerd.com, August 25, 2005. Accessed on February 6, 2006.

¹³ It makes sense to store rating data within the library rather than within each file, since they may be shared between different users on a machine or network, who are likely to have different ratings.

¹⁴ To alter the track end time in iTunes 6.0.2, use the Info window. Select the Options tab and type the value into the relevant box. This is also useful for modifying tracks that have a large amount of ‘quiet time’ at the start or end of them, like some hidden tracks on CDs.

¹⁵ I probably wasn’t planning ahead when I wrote the script to reset the play counts, but I’ll pretend I was.

¹⁶ The lack of a linear relationship between rating and play count surprised me, at least!

¹⁷ The indented lines starting with "&" are only broken for readability; they should be attached to the end of the previous line.

¹⁸ Actually, track 21 was played less than track 20, but only by around 2%.

¹⁹ Putting together a playlist of only half- and full-star songs was easy using a Smart Playlist. The Smart Playlist feature does not allow you to specify the inclusion of a song rated with a half star, but I was able to use the song title instead, as it was same as the rating on the 0-100 scale.

²⁰ Since there are only 11 tracks in this experiment, I can easily include the raw data. Unfortunately, I didn’t record for how long the experiment using the data subset ran. Exercise for the reader: Calculate approximately for how long this experiment ran from the data contained within this article.

²¹ Believe it or not, I am actually a Woodwork Teacher. I’m only a Nerd in my own time.

²² Python Software Foundation. Python Language Website. Python.org, Accessed on 10 Feb, 2006. A terse yet powerful open source programming language.

²³ Tim Rogers & the Twin Set. The Songs They Played As I Drove Away. What Rhymes With Cars And Girls. MSI/BMG. 1999.

²⁴ Paul Kelly. If I Could Start Today Again. …Nothing But A Dream. Cooking Vinyl. 2001.