EG: a month old child with 102 fever needs urgent attention compared to a ten year old.
If the child looks sick to you and or not acting her normal, yes go to ER or doctors office.
You could give sponge bath with tap water ( not ice cold water) this helps evaporate the heat.
Do not give aspirin in any form.
Liquid Tylenol or liquid ibuprofen is ok.
Best advice I can give you is to get the child seen by a physician.
The reason could be viral or more serious bacterial infection.
Early better than late, get to the doctor.