What is a general process of making a GUI in a game without resorting to a full blown GUI library? What I mean by this is how would I go about constructing a GUI layout like those in Java that are not reliant on pixels at a certain app state and are not only dependent on common presets like centering, right, or left aligning of an image via the window size?
Usually, elements in graphical user interfaces are laid out in a hierachical fashion, e.g. all the elements are children of the window or another element.
In most ui systems I've worked with so far, the parent element is responsible for layouting the children.
But let us digress to event handling first: When a user clicks somewhere on the gui, the window (respective the root element) receives the event and hands it down to its children, which either process the event (e.g. a button could fire its 'I HaveBeenClickedOn'-event) or hand it further down. When no child processes the event, the root element processes the event or discards it.
Most guis are modular. They're also mostly made from heavily decorated rectangles.
The most simple ui element is a label, which is just a line of text displayed at the specified position (depending on the renderer used, this could be actually rendered text or just a prepared texture with the text).
Next up is the button - this is basically a label with a (rounded) rectangle drawn around and the abillity to fire an event when a user clicks inside the buttons area. Many ui libraries allow the user to add an image to buttons, or to use an image instead of the text and outline.
(As I will mostly deal with Layout in the rest of the answer, I won't go into other ui elements. An overview over commonly used elements can be found in http://qt-project.org/doc/qt-4.8/gallery.html and/or http://algo.math.ntua.gr/~symvonis/other-material/java_material/JavaTutorial/ui/features/components.html)
So, how does a gui layout the components? Let's say we've got, for example, a simple dialog, which might be made from a window, which itself contains a vboxlayout, which contains a label and a button.
The simplest layout is the hboxlayout (respective vboxlayout): it divides the available space horizontally (respective vertically) by the number of child elements, providing each child with an equal amount of space.
Going back to the dialog example: Assuming our window sits at (100px, 150px) and has a width of 300 pixels and a height of 42 pixels. The Vboxlayout takes this space, halves it vertically (because there are two children, label and button) and sets the labels position to (100px, 150px) and its size to (100px, 21px) and sets the buttons position to (100px, 150px + 21px = 171px) and its size to (100px, 21px).
A thing to note here is that many Interfaces work with invisible panels. Say we wanted to expand the dialog to have not one button, but two (e.g. ok and cancel). We could add a panel and a hboxlayout, so our dialog would look like this:
The label would get the same position and size as before, but the lower space would be divided further by the hboxlayout.
Of course, this is a gross oversimplification. Most ui systems provide the abillity to add space between elements (e.g. padding and spacing) and/or allow the user to set the minimum and maximum size of objects. For an overview over more advanced layouts, you might want to look at http://docs.oracle.com/javase/tutorial/uiswing/layout/visual.html and/or http://qt-project.org/doc/qt-4.8/layout.html for examples.
While more difficult to implement, they all have one thing in common: Layouts divide the available space according to some rule, then assign their direct children each to one of the resulting spaces. (Unless it's a grindlayout/gridbaglayout and you're allowing elements to span multiple rows/columns)