You can apply TRUNC & TO_CHAR like this:
CASE
WHEN date between TRUNC(CURRENT_DATE-1, 'Q') AND CURRENT_DATE-1
THEN TO_CHAR(CURRENT_DATE-1, '"Q"Q"_"yyyy')
WHEN date between TRUNC(ADD_MONTHS(CURRENT_DATE-1,-12), 'Q') AND ADD_MONTHS(CURRENT_DATE-1,-12)
THEN TO_CHAR(ADD_MONTHS(CURRENT_DATE-1,-12), '"Q"Q"_"yyyy')
END
You can apply TRUNC & TO_CHAR like this: