You can use the 3-4-5 method of creating right angles.
Begin with a construction line and mark a zero point and a 3 point. Make a mark. Notice that I'm not using units of measure. You can use millimeters (preferred) or inches or anything in between.
Draw from the zero point at a right angle as closely as possible. Measure from the zero point to a 4 point. Make a mark.
Measure from one of the marks to the other and it should be a 5 point. If so, you have your right angle.
If not, you will have to erase or otherwise ignore the second line and create yet another.
I prefer using metric measurements for this type of project. You can adjust the numbers to fit your workspace. Let's say that you have a small piece of paper on which you wish to create your square.
The first line could be 90 mm long (3 * 30), while the second line would be 120 mm (4 * 30) and the points should be 150 mm apart (5 * 30).
Instead of drawing the second line as described above, consider to have a piece of paper with the length necessary for the conditions (120 mm) and position it with the zero point on the first line. Move the paper in an arc until it meets with your 150 mm point on the ruler. You can then mark the base paper with your points for a perfect square without having to erase misplaced lines.
As noted in the image below, greater accuracy is gained with larger distances. The quality of the measuring instrument also plays a factor, but only a minor part.