We ran into the same thing on a D7 project. Our solution was to add an extra field "Paragraph name" to the paragraph. In the paragraph display there is a view for Paragraph Editor Preview. Display that field name only there. You could also add another field, if needed, for a description (if you wanted to display more info), or anything else. Then set the paragraphs to preview as you described.
We ran into the same thing on a D7 project. Our solution was to add an extra field "Paragraph name" to the paragraph. In the paragraph display there is a view for Paragraph Editor Preview. Display that field name only there. You could also add another field, if needed, for a description (if you wanted to display more info), or anything else. Then set the paragraphs to preview as you described.