Yes, the toxic released are dangerous to fishes.
However, if your tank is made up of peaceful inhabitants then it should not be a problem. In ocean, sea cucumbers release their internal organs which are toxic when they are attacked or threatened by predators and they will eventually regenerate their organs. But in tank, most likely they cant recover.
If your tank has fishes that will typically pick on feather dusters then maybe the chances of attacking the cucumber will be higher, eg. butterflies and angelfishes. Frankly these are from my read-up, I have not kept them before and I am also interested to get a very small one to clean the sandbed.