Inside the Nova service framework

In my previous spelunking article, I went over the basic pieces needed to get a nova service stood up. Well okay – I skipped logging – maybe another article later for that later…

Quick recap: The service framework in nova is set up to make it easy to write your own services that interact with any other nova services (such as nova-network, nova-scheduler, etc). The service framework includes all the pieces to communicate with these other services (using a module called rpc.py that abstracts away the communications), a database connection for looking up data from the nova persistence store, and leaves the rest to you.

The service module is expecting to be told a manager class to load, and the framework will use that to do what it needs. There are only two required methods to overwrite:

• init_host
• periodic_tasks

There are two kinds of services – I’m going to focus on the stand-alone (not WSGI) service that expects to communicate and respond entirely through the message queue system in Nova.

So how does this service critter work? Well, when it’s initialized, it gets a number of attributes assigned to it. This is typically done from a class method on service.py:

from nova.service import Service
my_service = Service.create()

The create() method has a number of parameters:

• host – a string with the host this service is running on
• binary – a string with the binary name of this program
• topic – a string with a subset of the binary name, used to set up a message exchange in AMQP
• manager – a string with the class name to be loaded to do the ‘work’
• report_interval – an interval set from Flags, triggering a regular reporting loop
• periodic_interval – an interval set from Flags, triggering a periodic loop to do repeating tasks

If you don’t provide any of them, all these values get populated with defaults and from configuration detail (the flags) in the service.serve() method. Once serve() gets everything configured up, it calls .start() on each service to kick it into gear.

start() is where things really get moving. This is where the service manager class gets loaded, registered with the nova database (if it isn’t already), and the RPC mechanisms get spun up with Eventlet greenthreads to accept messages to this service. The topic (which is the name of the binary, minus any “nova-” in front of it) is used as an exchange. Through this mechanism, any service can talk to any other service (or set of services). Here’s how that works:

rpc.py has two methods: call() and cast() that do all the heavy lifting. When you use these, they take a “context” (i.e. authorization for who’s doing the call), a topic (the name of the service you’re calling), and a message. call() sends this message and waits for a response. cast() sends the message entirely asynchronously, not expecting a response.

The message is a JSON structure – a dictionary, and it’s expected that the dictionary will have a key “method” and another key “args”. method is expected to be a string, and args is expected to be another dictionary. The rpc module does the work of using that method string to look up and invoke the method on your manager.

An example of this operating is right in the code. In the nova-network API, there’s an rpc.cast():

rpc.cast(context,
         self.db.queue_get_for(context, FLAGS.network_topic, host),
         {'method': 'associate_floating_ip',
          'args': {'floating_address': floating_ip['address'],
                   'fixed_address': fixed_ip['address']}})

Through the service framework and the RPC mechanisms, this is calling associate_floating_ip() on the network service manager class.

The other nifty thing about service is that it’s keeping and managing a number of greenthreads from Eventlet to do it’s work. The basic bits are all encapsulated in that rpc.py mechanism – when it sets up connections to the message queue service to receive communications, that starts a greenthread rolling to watch out for, pull, and process any messages inbound. The two periodic interval pieces are also spun up on their own greenthreads – looping every “interval” (specified by the flags –report_interval and –periodic_interval, set at 10 and 60 seconds by default respectively). These run continuously until the service is terminated.