I've created a survey and changed the submit button:

When previewing the survey it appears as it should:

But when the survey is run by end users, it defaults back to "Submit":

Is there an option that I am missing to change the value, or is this a bug?