You can determine if the colony is yeast or bacteria by looking at the color and shape of the colony. Bacteria colony is usually yellowish and the shape is smooth.
You can also use a pipette tip to touch the colony, drop it into water and transfer some to a microscope slide. Observe the colony under light microscope. Bacteria is much smaller than yeast. E.Coli is rod shape. Budding yeast has buds.