The Bible doesn’t say anything directly about this; the problem didn’t arise because Christians were few in number in those days, and they knew they needed to draw strength from each other.
If it’s at all possible, however, I certainly urge couples to attend the same church. Not only will it help them grow closer to each other and to God, but it will help their children learn about Jesus. Although my wife and I came from different denominations, we happily attended the same church in our small community.
However, I know it’s not always possible for couples to agree on this, and rather than cause discord, on rare occasions it may be better to take the path you’ve suggested. However, don’t do it casually or simply because it’s easy. Pray together about it, and ask God to guide you and help you know His will.
Don’t judge any church too quickly, but ask God to guide you to one where the Bible is preached and taught, and you can have fellowship with other believers. In addition, think through what you really need in a church. Do you simply want a church that reminds you of the one you grew up in? Or are you honestly seeking a church where you can grow in your faith, and you can serve Christ?
Above all, put Jesus Christ at the center of your lives — as individuals and as a family. When our lives are centered in Him, disagreements begin to fade. Remember the Bible’s admonition: “Let us not give up meeting together… but let us encourage one another” (Hebrews 10:25).