Acetone is the well-rounded budget conscious option. Zizouz212's answer generally covers this. It is hardly the only option though. It largely depends on the material the glue is on well.
When it comes to skin (not a material but certainly somewhere glue ends up) warm soapy water is certainly effective. How long you scrub or immerse depends on the amount of glue. Getting it on your fingers is common and in some cases you can just let it harden and it will peel off. This is where I get CA glue stuck often and I just peel off what I can and wash the remainder.
Using acetone on plastics, with the intent to remove just the CA glue, is a very bad idea as it can dissolve/damange many of them. Acetone is one of the ingredients used in plastic welding applications if it puts that in perspective.
There are many commercial debonders that are made for this purpose. An example would be Un-Cure CA Debonder. You should know the materials you are working with to best match your debonder. Depending on your project these can be more effective but, again, have a higher cost then on the shelf acetone/ nail polish remover.
Wood, especially end grain, will actually absorb CA glue. If you actually needed to remove the glue you would have to remove the affected wood. Depending on the project you could use plastic wood filler or glue in new wood. This advice is not specific to CA glue but most glues.
Clothing is a broad topic as well as there are a multitude of different materials so there is no guaranteed approach. Acetone is generally recommended here but try to test it in a non visible area to ensure that it would damage/stain the fabric. Letting the affected area sit in some water and laundry detergent can also help. Using a toothbrush to massage the area and try to get the glue to flake off. Some debonders also work on fabrics as well keeping in mind the same caveats for clothing.
In general CA glue should be easy enough to remove. If there is concern, wear protective clothing and be aware of the specific materials that could come into contact with CA glue so you know the best approach to deal with it.