I'm encountering an issue with Facebook's Graph API where it inconsistently scrapes Open Graph (OG) meta tags between different language versions of my website, which is built using Next.js version 14 with an architecture using page router. Specifically, the French version of my URLs (manage by a context) consistently returns all expected OG meta data (title, description, image), whereas the regular English version sometimes only returns minimal data (type, updated_time), missing critical tags like title, description, and image.

Here are the key details:

Platform: Next.js version 14 with dynamic content rendering based on user language preferences, which is managed by a context.

Behavior: When using Facebook's Graph API Explorer or the Sharing Debugger, the French URL (e.g., https://wiseduckdev.vercel.app/fr/gpts/framework) always successfully returns full meta tag information. However, the equivalent English URL (e.g., https://wiseduckdev.vercel.app/gpts/framework) often only returns partial data.

Meta Tags: Both versions use identical setups for Open Graph meta tags, confirmed via direct source inspection and tools like SEO Meta in 1 Click.

 <meta property="og:type" content="website" />           <meta property="og:title" content={pageData.category.og_title} />           <meta             property="og:description"             content={pageData.category.og_description}           />           <meta             property="og:url"             content={               activeLanguage === "FR"                 ? `https://${siteUrl}/fr/gpts/${pageData.category.category}`                 : `https://${siteUrl}/gpts/${pageData.category.category}`             }           />           <meta             property="og:image"             content={`https://${siteUrl}${pageData.category.og_image}`}           />           <meta             property="og:locale"             content={translation.og_locale[activeLanguage]}           /> 

Caching: I've purged both server and Facebook caches and forced re-scraping using Facebook's tools, which sometimes resolves the issue temporarily for English URLs but doesn't provide a consistent fix.

I'm looking for insights or suggestions on why this discrepancy might be occurring and how to ensure consistent meta tag scraping by Facebook across all language versions of my site.

Actions Taken to Resolve the Issue:

• Meta Tag Verification: Confirmed that both the English and French versions of the site have correctly implemented Open Graph (OG) meta tags (og:title, og:description, og:image) using direct source inspection and SEO Meta in 1 Click.
• Caching Management: Purged server and Facebook caches to ensure fresh data is being fetched and used the Facebook Sharing Debugger to force re-scraping of the URLs.
• Server Response Checks: Used curl to simulate Facebook’s scraper requests, verifying that the server sends the correct metadata and appropriate HTTP responses.
• Dynamic Content Handling: Ensured that metadata is dynamically generated based on user language settings and is properly rendered server-side in both language versions.
• API Utilization: Utilized Facebook’s Graph API Explorer to manually fetch data for both language versions to check the responses and confirm that the issue persists despite correct setup.

Expectations and Goals:

• Consistent Meta Tag Scraping: Ensure that Facebook’s scraper consistently retrieves all relevant OG meta tags across all language versions of the site, not just the French version.
• Understanding Root Cause: Gain insights into why the English version sometimes only returns minimal data (type, updated_time) despite identical setups.
• Reliable Resolution: Find a reliable solution or workaround that ensures all versions of the site are equally optimized for social media sharing, without needing to manually trigger re-scrapes.