You're on the right track using a craft knife or utility knife. They get you started, but you need to finish them off with another tool to make them smooth.
Make sure your blade is extra sharp. If you have to put too much pressure to cut down the point, you'll be stressing the insides of the pencil. It may not break charcoal this time, but stress fractures could make a future sharpening very frustrating.
Once you've got a suitable length of your charcoal/pastel/graphite, you finish it off with a lead pointer or sanding stone. The only difference between graphite and the more brittle materials, from what I found, is how lightly you need to sand them.
Here's a pointer that's almost identical to the one I have:

You can make one yourself very easily if you have fine sandpaper lying around. It's just several sheets of sanding paper layered, so that you can peel them away when they get worn out. (Don't toss them just because they get dark/colored, as that can be tapped or blown out. I toss mine when they loose their grain.)
You can also buy small sanding stones, which is what they use in this video about sharpening General's charcoal pencils.
By sanding it, you'll be able to develop the exact tip you want. You can make it very pointed and conical, or you can sand it to be a sharp wedge, or whatever you need for your strokes.