FLV videos with alpha channel transparency are supported in Articulate Storyline's Flash and Articulate Mobile Player (AMP) output. See this article for a comparison of Flash, HTML5, and AMP features.
There was an issue in earlier builds of Storyline where alpha channel transparency wasn't working as expected in published courses. This issue was corrected in Update 5 for Storyline. To take advantage of the new features and fixes, install the latest version from the link in your product confirmation email or download it here.
After installing the latest software update, delete the affected FLV videos from your project file, then insert them again and republish your course.