Starting in VDP 6.1 there is a nice new feature where if quiescence fails it will fall back to a non-quiesced backup. But I believe the check box does opposite of what you would think. Maybe this could be fixed in the next release.
When its unchecked VDP will do a non-quiesced snapshot if quiescence fails. If its checked it does not fall back to non-quiesced snapshot.
I confirmed looking at the details of the dataset:
When its checked - type=boolean:name=fail_on_quiesce_error:value=true
When its unchecked - type=boolean:name=fail_on_quiesce_error:value=false