You'll need to review Joomla documentation below regarding Usergroup usage as is far too complex for me to explain simply if you've no experience with using usergroups.
docs.joomla.org/
Can i group users...into different user groups manually....if their is no field in there profile to make them relevant in that group?
Yes, by default they're given the Registered usergroup. You can edit your users within CB > User Management to change their usergroups.
Rephrase = do userlist depend on a profile field to auto put them into that userlist----or is it other things or a combination
Your users need to be in the usergroup(s) you select to display. This is the "User Groups to Include in List:" parameter. This is explained within your documentation as well in addition to being explained directly next to the parameter on the right (parameter description) when editing your userlist.