Pages

11/15/2011

Attempt to use Siri on Ubuntu

First off, according to this post from Applidium http://applidium.com/en/news/cracking_siri/
It is about what they learn while trying to figure out how Siri works.
They also provide tools that they used during the process here https://github.com/applidium/Cracking-Siri
and they were written in Ruby.

So, I thought,... Well, it's a good time to try it out. Since I am not an Apple fan boy, I have never use Siri before and I don't have iPhone. All the things matches up with my situation, I then started to try it out.

Unfortunately, after awhile of trying to make it works, I can't.
It seems this tool works by creating a proxy server to capture the authenticated session, use "an actual iphone" to communicate to Siri server to get a session, then you can use that session with the client tools they provides....

So... it  needs an iPhone, and I don't have one....
This tool is not working for me (yet),
but I hope it will work with others.
(or until I can find an iPhone to test it)



Here is the steps I tried:


1. Install all the requirements
It needs ruby, rubygems, openssl, libssl-dev, ffmpeg, speex, eventmachine, uuidtools, CFPropertyList
to install

sudo apt-get install ruby rubygems openssl libssl-dev ffmpeg speex
sudo gem install eventmachine uuidtools CFPropertyList

Possibly also need Audacity or other recording tool to create a sound file

2. Download the tools and extract them
p.s. you can read README.md and Siri.txt inside the first file, but I think it is more like a summary (doesn't give much detail).

3. Generate a certificate and such
(for more information about .cer, .crt, .key, PEM, DER read this http://www.gtopia.org/blog/2010/02/der-vs-crt-vs-cer-vs-pem-certificates/)
I found this command below from http://shib.kuleuven.be/docs/ssl_commands.shtml and only one line it can create both .crt and .key for the certification. I should work (remind that I don't have an iPhone to test)

openssl req -x509 -new -out MYCERT.crt -keyout MYKEY.key -days 365

It will prompt several questions for you to answer. Answer them whatever, but you need to
  • type in the passphrase, it can't be blank
  • "Common Name" (CN) answer as "guzzoni.apple.com" (without quote)

After the prompting questions stop, you will get file MYCERT.crt (for certification, signed) and MYKEY.key (for key)

4. Put the certificate to iPhone 
Well, I don't have one, so I'll skip it.

5. Configure Siri local server
File "siriServer.rb" contains code to work as a Siri proxy between your client (iPhone or Ruby) and Apple's Siri server.
It needs to be configured to use the certificate and key that we created.
First, find the lines below 
Default file names from the tool are "server.passless.crt" and "server.passless.key". 
They need to be changed to the files we just created (or vise-versa).
If you choose to edit the server file, it would be...

6. Start the server
This server will act as Siri server, which is using HTTPS connection. Therefore, this Ruby server will bind to port 433 (default HTTPS) and it also needs super-user permission (sudo needed). 

sudo ./siriServer.rb

If it shows nothing but the waiting prompt, that means it is running correctly. It is waiting for a connection from a client.

7. Use iPhone to connect to Siri via our local server
I haven't done this, but I have some ideas on it.
In order for iPhone to use Siri, it will need to connect to "guzzoni.apple.com". We don't want it to connect directly to the Apple's server, but we need to route the traffic to go through our Ruby server first, so we can capture and view the data inside the traffic.
According to this http://stackoverflow.com/questions/2028544/does-hosts-file-exist-on-the-iphone-how-to-change-it , there are ways to run Siri via our server

  • Router configuration: if you have access to router admin page, and the router support DNS modifying.
  • Using another proxy server: if you can setup another proxy server (Fiddler for example. see http://conceptdev.blogspot.com/2009/01/monitoring-iphone-web-traffic-with.html) and set iPhone to use proxy server and set hosts file on that machine to our server. This will redirect the traffic through the proxy server and finally to our Ruby server. 
  • Edit /etc/hosts (for rooted iPhone): install "iFile" app from Cydia, then navigate to /etc and open "hosts" file to edit. (more about hosts file http://en.wikipedia.org/wiki/Hosts_(file))
  • Setting up a new DNS server: possible but it'd be a hassle.
After the connection setting is working, try using Siri with an app such as Notes.app , server will dump some data out (including session data) that we can use later
- Make Siri dictation request, for example from the Notes.app application. On the server, this will dump all the "interesting" bits (X-Ace-Host identifier, sessionData and such).

8. Configure Ruby client (for speech-to-text)
In the downloaded files, there is a file called "Siri.old.inline.rb", used for speech-to-text conversion.
This file takes

  • Input audio file name: In the file, "input.sif" is fixed as the input. Change it if you need it.
  • Session data: needs several values captured from Ruby server step 7. to replace the word "COMMENTED_OUT"
  • Siri server address: By default, it connect directly to guzzoni.apple.com, but you can comment out/change address to localhost, or other host.


If the session data values are captured, "theoretically" you do not need Ruby server and iPhone anymore. In this step you can put the captured data in this file, and connect directly to guzzoni.apple.com.

9. Prepare Speech file
  • First, prepare a sound file for any format. You can use the original speech file from Step 2. (recording.m4a) or you can use a software to record your own voice (I use Audacity to do it). 
  • Then convert it into raw format by using ffmpeg. Because the Speex tool has default input format as "raw PCM input is 16-bit, little-endian, mono", ffmpeg will have "-acodec pcm_s16le". Command use:

    ffmpeg -i recording.m4a -acodec pcm_s16le -f rawvideo recording.raw

  • Convert raw sound file into .sif:

    speexenc testsound.raw input.sif
10. Run Ruby client 
Simply use command
./Siri.old.inline.rb

It will load the input speech file (default=input.sif) and connect to Siri server, get data back and then dump the data on the screen.



Some More Screenshots

Server dumps data that client sending through it


Got this "Not authenticated" appears on client-side because I do not have session data


Errors I Found
  • "what(): Encryption not available on this event-machine"
    • Ruby's library "eventmachine" is required SSL library to connect to HTTPS. 
    • To fix it, install "libssl-dev" (Ubuntu package)
  • "eventmachine.rb:572:in `start_tcp_server': no acceptor (RuntimeError)"
    • Ruby's library "eventmachine" cannot bind to a port while creating a server, dues to permission
    • Use super-user (sudo) to run the program, e.g. sudo ./siriServer.rb

Test Environment:

  • Ubuntu 11.10 x64
  • Ruby 1.8



Hope I can complete this tutorial soon.
(if I can find an iPhone to test)



UPDATE (related links):



2 comments:

  1. Thanks for posting this! Setting up VM to test!

    ReplyDelete
  2. Hello, i have a problem, when i connect the siri to my "personal siri server"

    the server "./siriServer.rb" returns the following error, and crash:
    terminate called after throwing an instance of 'std::runtime_error'
    what(): Encryption not available on this event-machine

    ReplyDelete