This is normal. The sludge is oxidised hop proteins that often appear as a dark brown/black sediment on the sides of the fermenter at or just above the wort level.
To clean your fermenter, fill the fermenter with a warm cleaner solution (PBW or another similar cleaner) and allow to sit for 5-10 minutes based on how bad the sludge is. After letting the solution sit, slowly allow the cleaner to drain while using a dish brush to clean the sides. This will remove all the proteins from the fermenter so that it is ready to be used again.