1. The problem was not with the food in these Jew-Gentile fellowship meals (i.e. whether it was kosher or not). Although some Jews would never eat with a Gentile (e.g. Qumran, Acts 10.28; Jub 22.16) most Jews in the Diaspora did engage in some kind of interaction and association with Gentiles and without compromizing their Jewishness (e.g. Ep. Arist. 172-86; Josephus, War 2.461-63; 7.41-62; Apion 2.39). Larry Hurtado writes:
[A]lthough some Jews refused any meal with Gentiles under any circumstances, for many, probably most religious Jews in the Hellenistic-Roman period, eating ordinary meals with Gentiles was not an insuperable problem; any claims by scholars to the contrary are simply misinformed. In principle, so long as the food on the table fell within what was permitted for Jews to eat under Torah (e.g. no pork), and so long as eating did not implicate a Jew in participating in a feast in honor of a god (e.g. no libation of wine or consecration of meat to a god), there was no major problem. Second, Jewish Christians’ objections to eating with Gentile Christians in Acts (11:1-18) and Galatians (2:11-21) were not about what food was served, but about having meal fellowship with Gentiles whom they regarded as incompletely converted. This issue was not “purity laws,” but the requirements for treating Gentiles as fully converted to the God if Israel.
Larry Hurtado, Lord Jesus Christ, p.162, n. 18.
2. The real problem was not the food but the company it was consumed in. The objection of "those of the circumcision" and "certain men from James" was that these meals made Gentiles equals and not simply guests in the Jewish Christian community! Gentiles did not have to judaize/do works of the law/be circumcized in order to have the membership status and privileges of Jews. Circumcision of Gentiles is what links Gal 2:1-10 and the rest of Galatians with Gal 2:11-14. Paul Barnett writes:
The “truth” of the gospel was upheld when belonging to Christ was deemed sufficient for “righteousness” for God and for “inclusion” (proslempsis) in his covenant people (Rom 11:15). That “truth” was overturned, however, when the “works of the law,” including circumcision, as demanded as necessary for righteousness and inclusion. The “truth of the gospel” was under threat in Jerusalem when the “false brothers” attempted to impose circumcision on Titus, when the agitators in Galatia insisted on circumcision for Gentiles, and in Antioch-on-the-Orontes when, in effect, circumcision of Gentiels was made a condition of table fellowship with Jews.
Paul Barnett, “Galatians and Earliest Christianity,” RTR 59 (2000): 124.