Building Firefox OS On OS X and Hamachi

At my first day at Mozilla, they gave me two brand spanking new Firefox OS phones. Of course, the first thing we do is build Firefox OS and flash the devices with the latest OS from the repositories. Of course, it's much easier said than done! There were a few extra steps that I needed to do to get the build working.

Preparing the Build
There's a nice link on what you need here. If you're lucky, you can just run B2G/scripts/bootstrap-mac.sh and be good to go. Then ignore this post :)

Firefox OS requires a specific version of gcc - 4.6.3, with custom patches, to build Firefox OS. Unfortunately, the bootstrap-mac.sh script only checks that you have gcc 4.6,  not necessarily 4.6.3. However, this version of gcc wasn't quite building correctly for me. As it stands, the clang that comes with OS X as well as the clang that is released on llvm.org cannot build the specific version of GCC. Only GCC can build the custom GCC!

The first step then, is to download the latest stable build of gcc. I used gcc 4.7 and recompiled and installed it by following the instructions. This version of gcc built just fine with clang 3.3. Next, I tried to build the gcc version used with Firefox OS, but I was getting a lot of errors. The main one was:

{standard input}:82:no such instruction: `vmovaps %xmm2, %xmm0'

This is because the homebrew script assumes you're using a newer Macbook Pro that supports the newer AVX instruction set provided by Intel. I had to modify the homebrew script to remove a few flags. If I did a brew --env, I'd get the following output:

brew --env
CC: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
CXX: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ => /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
CFLAGS: -Os -w -pipe -march=native -Qunused-arguments -mmacosx-version-min=10.8
CXXFLAGS: -Os -w -pipe -march=native -Qunused-arguments -mmacosx-version-min=10.8
LDFLAGS: -L/usr/local/lib
MAKEFLAGS: -j4
MACOSX_DEPLOYMENT_TARGET: 10.8
PKG_CONFIG_LIBDIR: /usr/local/lib/pkgconfig:/usr/local/Library/ENV/pkgconfig/10.8:/usr/lib/pkgconfig
OBJC: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
PATH: /Library/Frameworks/Python.framework/Versions/2.7/bin:/usr/local/bin:/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin:/usr/local/go/bin:/usr/texbin:/Users/masonchang/Projects/adt-bundle-mac-x86_64-20130911/sdk/platform-tools:/Users/masonchang/Projects/moz-git-tools:/Users/masonchang/Projects/llvm/build/Release/bin:/usr/local/Library/Contributions/cmd:/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin:/Applications/Xcode.app/Contents/Developer/usr/bin
CPATH: /usr/local/include

We can see a few things here that don't quite work well. First, the "-march=native" enables AVX features that might not be available on your computer unless you have the latest Intel processors. Next, we have a minimum OSX version of 10.8, which prevents Firefox OS from building because it'll use the 10.8 SDK libraries rather than the required 10.6 SDK libraries. I couldn't find anything on the home brew web page about changing default configurations, but we can modify the home brew formula for Firefox OS' gcc version to this: 

    ENV['CFLAGS'] = '-Os -w -pipe'
    ENV['CXXFLAGS'] = '-Os -w -pipe'
    ENV['CC'] = '/usr/local/bin/gcc'
    ENV['CXX'] = '/usr/local/bin/g++'

We change the CC and CXX flags to use our explicit gcc 4.7 rather than clang. Then we remove the CFLAGS and CXXFLAGS to be bare bones and not require a minimum OS X version or set the architecture. If we brew install gcc 4.6.3 required by Firefox OS, we should be good to go! 

Running the Build 

After these GCC steps, we should be able to run config.sh and everything should work. The next problem was while actually building Firefox OS, I got an error saying my python version was incorrect. I need python 2.7.3 but I had python 2.7.2. After some discussion with Gregor Wagner, installing the binary distribution of python 2.7.5 worked where as building Python 2.7.5 doesn't. After installing python, I was able to build. Woot, day one complete!

Getting Your Geolocation in Firefox / Firefox OS

Getting your Geolocation in a browser is a fairly easy feature to understand yet has quite a large amount of complexity to implement. First, let's go back to our starting point, we can build a web application that displays our geolocation using the Geolocation Web API. What happens underneath the covers in the browser level to actually get your geolocation?

We start by parsing some of the Javascript, and thus the SpiderMonkey VM. Since our application isn't performance critical, the JavaScript is actually interpreted in Interpreter.cpp. We hit the js::Invoke function to call our native function. A native function is a function built into the VM rather than another JavaScript function in the same script. By this time, the JavaScript interpreter has looked up the function and determined that to get the geolocation, it has to call a native C function in the VM. The function that's called is Navigator::GetGeolocation, which initiates a new Geolocation Object.

During the initialization phase of the Geolocation object, it attaches itself as an "observer", which is just a listener to a GeolocationServices object. The GeolocationServices object is what actually maintains all the information about our Geolocation. Everything discussed so far applies to both Firefox OS and Firefox Desktop. However, how we actually fetch our location now differs between the two platforms.

Firefox Desktop
Firefox Desktop on Mac is a little bit easier to explain. When Firefox requests for your location, it jumps to nsGeolocationRequest::Allow, which fires after you allow Firefox to use your location. Allow() then checks for a cached Geolocation. If it exists, it fires off a new event with your current Geolocation. This is an important underpinning in the browser and JavaScript in general - Everything is an event.

A main thread constantly checks a queue to see if an event has arrived. Once it has, it processes the event. This is called the Event Loop. Our Geolocation just creates a new event and submits it to the queue, where another thread will pick it up and process our location.

Now what happens if we don't have a cached position? Actually what happens is once we create the nsGeolocationService, Firefox uses OS X's CoreLocation service. CoreLocation is an OSX provided API to get your location. You supply CoreLocation with a callback function whenever you want to get an updated location. What Firefox does is tell CoreLocation to callback nsGeolocationService::Update, which calls nsGeoLocationService::SetCachedPosition and sets our cached location. Thus, Firefox actually always has your location due to CoreLocation. When Firefox requests your location, it just reads the cached location which is constantly being updated by OS X's CoreLocation. If you deny your location to a website, Firefox just returns an error and the website doesn't get your address, but internally, (I think) the OS X and Firefox have your location ready to reply.

Firefox OS
Firefox OS has mostly the same logic as Firefox Desktop except there is one big difference. Apps in Firefox OS, like apps on Desktop, are different processes, but core functionality such as getting your Geolocation belong to the core B2G process. Different apps talk to the core B2G process to request your Geolocation. Each app has it's own event queue that's being run.

When an app requests your location, it creates a new event requesting your Geolocation. Through IPC, the event gets pushed onto the main B2G's event loop queue. When the main B2G process handles your Geolocation request, it submits the position event back through IPC to your app. The app then gets a Geolocation update, which pushes another event to update whatever part of the app needed the location.

Finally, there is some part of the OS that has to actually call the GPS driver to get the location. I haven't found that part yet, but I'm on the hunt!

My First Firefox OS App

I just wrote my first Firefox OS App, and it was actually a lot easier than I expected it to be. That's partially the beauty of the web, where every app is just a container around a website. And partially a nice part about Mozilla - Everything is open source so Googling for an error usually comes up with a good answer. What I want to do over the next couple of days is walk through the deep stack that is a web browser - What happens if I request for my location? So first up, what happens at the high level? Let's build a basic Firefox OS app that gives us our geolocation.

First, we use the web geolocation API, which gives us access to our GPS coordinates. Let's start off with a simple website that asks for our location, then prints out the coordinates. Here's the source for our index.html:

<html>
  <head>
    <title>Get your Location</title>
  </head>
  <body>
    <script type="text/javascript">
    function updateGeoStatus(message) {
      var field = document.getElementById("geoStatus");
      field.innerHTML = message;
    }

    function successGeoLocation(position) {
      updateGeoStatus("Latitude: " + position.coords.latitude 
         + " Longitude: " + position.coords.longitude);
    }

    function errorLocation(error) {
      updateGeoStatus("Wrong position error. Error " + error.code + 
          " message is: " + error.message);
    }

    function geoLocation() {
      if ("geolocation" in navigator) {
        updateGeoStatus("Waiting on location\n");
        navigator.geolocation.getCurrentPosition(successGeoLocation, errorLocation); 
      } else {
        alert("No geolocation!\n");
      }
    }
    </script>
    <p><button onclick="geoLocation()">Get the location</button></p>

    <div id ="geoStatus">
      Hi World
    </div>

  </body>
</html>

If you save this on your server and just load it up in your desktop browser, it should work. Once we click the button to find our location, we get a message saying we're finding it, and once we resolve the location, we print the coordinates to the screen. Seems simple enough. Now how do we make this an app that we can actually install on a Firefox OS phone?

First, let's look up some documentation on building our application. We have to do two things: 1) Add a manifest file and 2) Add an installer. To add a manifest.webapp file, open up your favorite editor and create a manifest.webapp. The manifest file describes a few things about your app such as the name and where in the web your app exists. The best part is that Mozilla has a Firefox OS Manifest Validator to make sure you're doing everything right. Let's break down what's going on in the manifest:

{
  "name"            : "The Awesome App",
  "description"     : "Awesome League",
  "launch_path"     : "/~mchang/",
  "default_locale"  : "en",
  "developer"       : {
    "name"          : "Mason",
    "url"           : "http://masonchang.com"
  },

  "icons"           : {
    "16"  : "img/logo16.png",
    "32"  : "img/logo32.png",
    "64"  : "img/logo64.png",
    "128" : "img/logo128.png"
  },

  "permissions" : {
    "geolocation" : {
      "description" : "Testing Geolocation"
     }
  }
}

A manifest file is basically a JSON file with a few descriptions. You need your URL, Name, App name, and where the app lives relative to / on the home server.  Icons are your app icons in different resolutions. One interesting thing to note though is that the icons path is actually relative to /, even though we stated the launch_path to be ~mchang. This seems to be either a bug in FF OS itself or the online validator. I still have to investigate what's going on there.

Next you'll notice that we have a permissions JSON entry as well. This is required as geolocation needs to be explicitly enabled in our app in our manifest.webapp files. Otherwise, if you run the application, you'll get a permission denied error when your app tries to read the phone's location.

Second, you'll need to create another file that tells Firefox OS to install the app. Let's load up install.html!

<html>
  <head>
    <title>Hello Shu</title>
  </head>
  <body>

    <script type ="text/javascript">
      var manifest = "http://masonchang.com/manifest.webapp";
      var request = window.navigator.mozApps.install(manifest);
      request.onsuccess = function() {
        var appRecord = this.result;
        alert("Installation Success");
      }
      request.onerror = function() {
        alert("Error: " + this.error.name);
      }

    </script>
  </body>
</html>

Our install.html just has some JavaScript that loads up on page load, points to our manifest location, and tells the navigator to install the application.

Now we have 3 files. We have is our basic index.html file that is our actual app, the install.html file that tells Firefox OS how to install the app, and our manifest.webapp file and we have our first app! You can test the app in the Firefox OS Simulator or a real device if you have one. Point the simulator or device to your install.html file and you're good to go! Woohoo! Now, if you run the app, push the button, you'll get your geolocation in text! (Actually doesn't work in the Firefox OS Simulator, but it does on the device and desktop browser). Next blog post, how does Firefox OS actually get your location from your running JavaScript?