Yes, drain as much "supply side" water as you can, and closely monitor the temperature in the basement. It would need to drop a degree or two below the freezing point, and stay there for a while, before pipes would burst (32-F / 0-C is the freezing/melting point, so it's mostly slush not ice at that temperature; it has to form solid ice to expand).

Ideally, it's best to use compressed air to blow out as much water as possible. That's standard practice when winterizing cabins and RVs around here. A little bit of water in a line won't burst it, even if it does freeze solid. Drain the toilet tank too.

I don't know if you have a pressure tank. There's probably a drain valve on that, and it may be the lowest point that's easy to access. If there's a pump and piping above ground, and you can't drain them, positively isolate the power (isolation switch and breaker) and cover with cardboard, old blankets, etc. You don't want a fire hazard when the power comes back on. Impeller pumps usually have priming/drain plugs, and it's probably wise to remove these.

On the "drain side:" past_digger is right: the traps under the sinks, shower, and toilet can also freeze. You probably shouldn't drain these, because they seal off the noxious gases in your septic tank from the air in your home.

The antifreeze normally used for the drains in cabins/RVs is non-toxic, unlike the glycol in your car rad. I keep a couple of jugs around just in case.

In a pinch, you could pour in a bunch of alcohol, or steal a few cupfuls of glycol from a car. Anything to keep the water from freezing into a solid, expanding mass.

It's also worth noting that salt water freezes at a few degrees lower than fresh water. It's not a lot, but everybody has salt; and depending on your situation it may buy you a small safety margin.