That's a tension dial, for adjusting how taut the upper thread is held while sewing; the adjustments run from 1 (least tension) to 5 (greatest tension), and allow you to adjust the pull of the upper thread against the thread coming from the bobbin. You've already observed this, by noting the loops migrating to the top (too high) or bottom (too low).
Different fabrics will often need different tensions, but there aren't really hard-and-fast rules for this. The typical process instead is to sew a test seam using the same fabrics and check your tension, adjusting up or down as needed until the two threads are balanced. That said, if you're machine basting or stitching a gathering seam, throwing the tension wildly off can make the basting much easier to pull and remove later, as one side will be very loose and easy to break (or pull through the loops of the other side, in the case of a gather).
Note that, while you can also adjust the bobbin tension in front-loading machines, this is rarely necessary and typically only done if something has gone wrong in the bobbin case. Most top-loading machines don't have a way to adjust the bobbin tension. Ordinarily, you'll only ever need to adjust the top thread tension.
Threads Magazine has a good illustrated write-up with information on factors other than the tension dial that may throw off your tension and necessitate additional adjustments.
