I also (pretty much) always have my dog with me.
As Benton says, probably the biggest thing is to have enough water for the dog and some way for the dog to drink it. I have a big (80lbs) dog and she gets hot. I pretty much consider she needs as much water as I do, which means I'd prefer to have 2L for each of us at all times.
For the car and office kit, that isn't a problem, but it is too heavy to have that with me at all times. My approach is to have water ready to go in my office kit so that I can just toss the water into my pack and go.
In my case, a bowl isn't a huge issue since I carry plastic sandwich (storage actually) bags to pick up after her. In a pinch, you just make an improvised bowl. I also use a titanium camping "pot" to keep my dust masks from getting crushed which I could also use as a dog bowl. In my car kit I have one of those collapsible dog bowls.
In general, what you have for your dog will depend on your situation. I live in earthquake country, so I think about rough terrain (debris) and building escape. As such, a
rescue sling seems like a useful item to have.
I also keep a set of dog boots for rough terrain, but I do worry about them impeading her ability to sweat and cool.
The other issue I worry about which I don't have a solution for yet is dust. I understand that several of the dogs that responeded to the 9/11 rescue effort died later due to gunk they breathed in. When I heard that, it made sense, but it (like other things prompted by 9/11) up until then I hadn't thought about it.
I wonder if an escape hood like
this one might work...
Oh, you want to make sure you have a leash (or way to improvise one). Personally, I like
this lead from Ray Allen because it has a ring in the handle so you can clip the lead back onto itself and wear it securely around your neck. It also small, and the braided portions allow for a comfortable, secure grip. The other nice thing about the ring on the handle is you can wear something like the Maxpedition
Key Keyper on your belt, and then have an easy way to clip the dog to you when you want to have your hands free.
-john
Edit: Good point Dave. Dog should have collar (with ID)! Mine is "chipped" as well. FAK is also importaint, but my overlook this a bit since I just factor this into my normal carry FAK.