No, you need to check the battery under load; merely connecting a multimeter won't do that. The idea is to see what happens to battery voltage while it's subject to a heavy current draw. A bad battery can easily show acceptible voltage when not under load, but drop way too low when a load is applied.