Both apples and honey together and separately have become symbols of the new year. They are used in cooking and other High Holy Day imagery.
Dipping an apple into honey is also an unusual act, one that we do not usually perform at other times of the year. By doing so we set apart the holy days from the ordinary days.