The Grim Underworld is a world parallel to Gielinor. Apparently, the souls of those who die in Gielinor are ferried to their final resting place, the Afterlife, within the underworld. Each realm has their own underworld, with the Gielinorian one called the "Grim" Underworld. When creatures die, their souls are reaped by Harold Death Esquire, a Guardian of Guthix, hence his nickname "the Grim Reaper", and escorted to the underworld. All underworlds, including the Grim one, have in common that whoever is the first to ever die on that realm becomes the escorter of the dead, as was the case with Harold Death.
Characters who die normally pass onto the grim underworld to be escorted to the Afterlife or stay and become ghosts if they have unfinished business or died a violent death.
The Afterlife on the other hand is a place where the souls of the dead go after being reaped by their realm's Death. The afterlife that the deceased goes to depends on what the departed considers to be their home, with each world having its own afterlife. The souls of Gielinorians go there after being reaped from their bodies by Harold Death, ferried over the River Noumenon by a boatman and guided through the Grim Underworld by Icthlarin, in that order. The Fremennik are an exception, having two different underworlds of their own.