Yes, but they don’t have to be.

In screenplays, it is up to your discretion and preference. But be consistent with your choice.

In TV, it depends on how the specific show chooses to do it. For multicam sitcoms, underlined scene headings are common.