As I mentioned there is much ado about nothing in the scholarly treatment of the infancy narratives because of the fact that the two evangelists, Matthew and Luke handle the story so differently. However there are no real conflicts in the two stories especially when you realize that that stay at Bethlehem could easily have extended over a period as long as two years prior to the flight into Egypt.
In reading the two treatments a direct harmonization can be accomplished in the following fashion:
Lk 1:26-38 The Account of the Annunciation and Mary is told that Elizabeth is pregnant.
Lk 1:39-56 Mary Visits Elizabeth staying about 3 months (Mary 3+ months pregnant)
Mt 1:18-25 The Birth of Jesus Christ (Matthew's account up to Joseph accepting Mary into his house)
Lk2:1-7 Journey from Nazareth to Bethlehem for the census and the birth of Jesus
Note that here we have the first apparent conflict between the two accounts. Luke begins in Nazareth and sends Mary to Elizabeth back to Nazareth thence to Bethlehem for the census. Matthew by contrast seems to begin in Bethlehem and stay there. He is actually silent about their location until after the birth of the child.
Lk 2:8- 20 The Shepherds and the Angels (Matthew is silent about this)
Lk 2:21-38 Jesus Presented in the Temple (Birth + 8 Days) (Matthew is silent about this and the prophecies of Simeon and Anna)
Mt 2:1-12 The Visit of the Magi (Birth +|- up to two years probably a year)
To reconcile the two accounts one only has to assume that Joseph and Mary stayed on in Bethlehem which is near Jerusalem for a year or so while the child was small. This is certainly reasonable since one does not like to make long journeys with an infant if one doesn't have to, especially on foot. Since Herod ordered male children up to 2 years of age killed it is clear some time had passed since the Magi had seen the star and Herod was uncertain of the child's age.
Mt 2:13-18 The Escape to Egypt (Joseph in accord with an angelic visitation in a dream flees with Mary and the child to Egypt) by now the child is between one and two and likely can walk.
Mt 2:19-21 The Return to Nazareth ( Circa 4 BC when Herod Died) and finally we can conclude with the passage from Luke:
Lk 2:39 39When Joseph and Mary had done everything required by the Law of the Lord, they returned to Galilee to their own town of Nazareth. 40And the child grew and became strong; he was filled with wisdom, and the grace of God was upon him.
It is reasonable to ask where the difficulty is? There are several usually cited. One is the dating of Quirinius as governor of Syria when this first census took place. I won't bother with that, but Quirinius was in the area in the proper time frame and it is not difficult to harmonize all the dates. The only other questions involve the silence in Luke about the Magi, Herod's attack and the flight into Egypt. Some of these fit Matthew's plan and ideas about what is significant more than Luke's. Luke is writing to Gentiles who don't care much about Jewish customs and ways while Matthew is writing earlier and trying to link Jesus to prophecy. He may know about their original home in Nazareth and yet also know they had lived a significant time in Bethlehem and the Messiah must be born in Bethlehem of Judea. It may be as simple as that.
So there are not very many problems. I heartily recommend the book The Mother of Jesus in the New Testament by John McHugh for those who seek more reading on the general subject.