There are three key variables to consider with this problem: fiber content, fabric construction, and dyeing method.
The fleece fabric construction is sort of a worst case because there is so much surface area for the adhesive to stick to that you won't be able to freeze the fabric and try to scrape it off - which is a good technique for getting sticky stuff like gum or adhesive off of flat fabrics.
If the fiber content is polyester, you probably can safely use a goo-be-gone product because it's most likely that the fiber was solution dyed, which means the dye is actually part of the fiber, not an add-on. (Dye is added to the solution before being extruded into a fiber shape.)
Even though goo-be-gone should be OK to use on polyester fleece, it's safest to apply a little with a Q-tip somewhere that doesn't show, like the inside of the side seam.
If the fleece is a natural fiber like cotton, you definitely need to do a test first because the dye will likely be less stable.
Textile dyeing can be conducted at the fiber, yarn, fabric or garment stages. The earlier in the process the dyeing occurs, i.e at the fiber or yarn stage, the more stable and durable. However, fiber and yarn dying are typically more expensive processes than fabric or garment dyeing so it's done less often.
Possible complication: Sometimes black fabrics of any fiber content have been overdyed - the fabric or garment may have been dyed a fashion color that didn't sell well, and then dyed black.
There is much about end-use textiles that is invisible to the human eye and to the average consumer. Without knowing fiber content and how the item was processed and dyed, there is always a risk when experimenting with chemicals on textiles.
Always test on a place that won't be seen, or take the item to a reputable dry-cleaner.