FrostWire for Android does support Playlists, but if you still do not have them set up, you can always listen to all tracks from the same artist one by one.
To listen to all tracks by the same artist, go to:
1. Tap on the hamburger icon to access Main Navigation
2. Choose My Music
3. Choose the Artists Tab
4. Locate the desired artist and tap on the name to access its contents
5. Start playing any file from the screen
6. The rest of the tracks from the artist should be now added to the play next queue
The FrostWire music player will play all the tracks found; in order they were found and will loop the playback of the entire queue of songs by that artist.