Large scale disk-to-disk backups using Bacula, Part V

The inte­gra­tion of Bac­ula into the rest of our com­pany has been a very straight­for­ward process.  Amongst other things we have an inter­nal con­trol panel that is used to con­duct most busi­ness oper­a­tions like cre­at­ing and chang­ing cus­tomer sub­scrip­tions  plus a billing com­po­nent which con­tin­u­ously mon­i­tors resource con­sump­tion and informs our cus­tomer data­bases about changes in ser­vice plans and so on.

Now, we wanted to inte­grated job track­ing into our con­trol panel. The details would be exposed on a per client basis, enabling both staff, resellers and part­ners to mon­i­tor and check the backup sta­tus with­out going through our front-line sup­port or both­er­ing our tech heads.  Fur­ther­more we wanted to be able to check resource con­sump­tion (disk usage) on a per client basis to enable auto­mated billing. Cus­tom report­ing and trend­ing were also on our wish­list and so are nicely for­mat­ted email noti­fi­ca­tions for our customers.

Almost every­thing we needed was already in the Bac­ula Cat­a­log data­bases. The only thing we had to do was to add

cat­a­log = all, !skipped, !saved

to our Mes­sage resources in the direc­tor con­fig­u­ra­tion in order to store job logs in the data­bases as well. It took me about one day to research and less than a day to write a sim­ple REST API that exposes this data as XML or JSON, depend­ing on what the client wants. The Bac­ula DB schema is rather sim­ple once you find your way around. Inte­gra­tion into the con­trol panel was done in lit­tle more than a day.

An exam­ple of the gen­er­ated XML can be seen here :

  <start­time>2010-12-14 02:22:37</starttime>
  <end­time>2010-12-14 05:39:15</endtime>
  <job­sta­tus­long>Com­pleted suc­cess­fully</jobstatuslong>

We will expand this in the future. I already have a pro­to­type that enables us to cre­ate, mod­ify and remove clients and their con­fig­u­ra­tion as well as restart­ing failed job and so on  directly from our con­trol panel . It also uses col­lected met­rics  in order to allow the sys­tem to place new clients on the least utilised direc­tor / stor­age servers automag­i­cally, thus enabling an equal dis­tri­b­u­tion across our Bac­ula infrastructure.


Part VI is now online

No Comments

Leave a Reply

Your email is never shared.Required fields are marked *