i202 Fall 2010 School of Information, UC Berkeley

 In today's lecture, Bob mentioned the differences between traditional linguistic and statistical approaches to language processing.  In my undergraduate linguistics major, I studied both syntax and "grammar engineering," for which we had to construct representations like the one below.  If anyone ever had to diagram sentences in middle or high school, it's kind of like that on steroids: 

Bob mentioned the usage of Latent Semantic Analysis (LSA) to do automated essay scoring. Surprisingly, the scores of the tool (IEA) agreed (a.k.a., were as good and as bad) with human experts "as accurately as expert scores agreed with each other". Had I known this before, I probably had spent more time doing text analysis of "good essays" than writing sample essays for my GRE test!

If you're interested in this stuff, take a look at the paper here: http://www-psych.nmsu.edu/~pfoltz/reprints/Edmedia99.html

Having talked about how the Enigma encryption algorithm and the analysis of language, it is interesting to note that Caesar already used a character shift encryption method to secure his messages from other unauthorized people. As a response, his enemies started to analyze language and figured out the frequency of characters in the language (as Bob has mentioned several times). In that sense, the analysis of language already goes back to Caesar... http://en.wikipedia.org/wiki/Caesar_cipher