Use resolve for OG metadata #6787

Merged
btzr-io merged 7 commits from fix-og-meta into master 2021-08-07 13:53:49 +02:00
3 changed files with 94 additions and 68 deletions

View file

@ -18,6 +18,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
- Improve twitter share _community pr!_ ([#6690](https://github.com/lbryio/lbry-desktop/pull/6690)) - Improve twitter share _community pr!_ ([#6690](https://github.com/lbryio/lbry-desktop/pull/6690))
- Update lighthouse search api _community pr!_ ([#6731](https://github.com/lbryio/lbry-desktop/pull/6731)) - Update lighthouse search api _community pr!_ ([#6731](https://github.com/lbryio/lbry-desktop/pull/6731))
- Update sockety api _community pr!_ ([#6747](https://github.com/lbryio/lbry-desktop/pull/6747)) - Update sockety api _community pr!_ ([#6747](https://github.com/lbryio/lbry-desktop/pull/6747))
- Use resolve for OG metadata instead of chainquery _community pr!_ ([#6787](https://github.com/lbryio/lbry-desktop/pull/6787))
infinite-persistence commented 2021-08-06 03:33:35 +02:00 (Migrated from github.com)
Review

Not sure where the error is, but Embed OG's aren't showing up (it's showing the fallback).
Example embed: 1fe325659b

Not sure where the error is, but Embed OG's aren't showing up (it's showing the fallback). Example embed: https://odysee.com/$/embed/eevblog-1412-argon-ion-laser-teardown!/1fe325659bf8d791f72e02bb614a942fa4b571bf?r=5yqd2A3cQ59DhkNRYTJq3BApzdcr7KSn
btzr-io commented 2021-08-06 04:15:52 +02:00 (Migrated from github.com)
Review

The embed url doesn't use : or # to separate the id.

The embed url doesn't use `:` or `#` to separate the id.
btzr-io commented 2021-08-06 04:17:24 +02:00 (Migrated from github.com)
Review

Adding this .replace('\/', '#'); fixes the issue. Not sure if this will work on all cases.

Adding this `.replace('\/', '#');` fixes the issue. Not sure if this will work on all cases.
btzr-io commented 2021-08-06 04:18:18 +02:00 (Migrated from github.com)
Review

As long as embed url is always $/embed/name/id ^^

As long as embed url is always `$/embed/name/id` ^^
### Fixed ### Fixed
- App now supports '#' and ':' for claimId separator ([#6496](https://github.com/lbryio/lbry-desktop/pull/6496)) - App now supports '#' and ':' for claimId separator ([#6496](https://github.com/lbryio/lbry-desktop/pull/6496))
@ -29,6 +30,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
- Fix yarn copyenv on windows _community pr!_ ([#6702](https://github.com/lbryio/lbry-desktop/pull/6702)) - Fix yarn copyenv on windows _community pr!_ ([#6702](https://github.com/lbryio/lbry-desktop/pull/6702))
- Fix unnecessary livestream api calls in channel page _community pr!_ ([#6652](https://github.com/lbryio/lbry-desktop/pull/6652)) - Fix unnecessary livestream api calls in channel page _community pr!_ ([#6652](https://github.com/lbryio/lbry-desktop/pull/6652))
- Fix desktop app fails to resolve deep links _community pr!_ ([#6779](https://github.com/lbryio/lbry-desktop/pull/6779)) - Fix desktop app fails to resolve deep links _community pr!_ ([#6779](https://github.com/lbryio/lbry-desktop/pull/6779))
- Fix wrong release date on GoogleVideo metadata _community pr!_ ([#6787](https://github.com/lbryio/lbry-desktop/pull/6787))
- Fix markdown line breaking mid word _community pr!_ ([#6805](https://github.com/lbryio/lbry-desktop/pull/6805)) - Fix markdown line breaking mid word _community pr!_ ([#6805](https://github.com/lbryio/lbry-desktop/pull/6805))
- Added \ and = to reserved symbol warning _community pr!_ ([#6733](https://github.com/lbryio/lbry-desktop/pull/6733)) - Added \ and = to reserved symbol warning _community pr!_ ([#6733](https://github.com/lbryio/lbry-desktop/pull/6733))
- Don't break words in chat + fix text overflow past 3 dot menu _community pr!_ ([#6602](https://github.com/lbryio/lbry-desktop/pull/6602)) - Don't break words in chat + fix text overflow past 3 dot menu _community pr!_ ([#6602](https://github.com/lbryio/lbry-desktop/pull/6602))

View file

@ -12,10 +12,13 @@ import Card from 'component/common/card';
import { formatLbryUrlForWeb } from 'util/url'; import { formatLbryUrlForWeb } from 'util/url';
import { parseURI, COLLECTIONS_CONSTS } from 'lbry-redux'; import { parseURI, COLLECTIONS_CONSTS } from 'lbry-redux';
const AbandonedChannelPreview = lazyImport(() => import('component/abandonedChannelPreview' /* webpackChunkName: "abandonedChannelPreview" */)); const AbandonedChannelPreview = lazyImport(() =>
import('component/abandonedChannelPreview' /* webpackChunkName: "abandonedChannelPreview" */)
);
const FilePage = lazyImport(() => import('page/file' /* webpackChunkName: "filePage" */)); const FilePage = lazyImport(() => import('page/file' /* webpackChunkName: "filePage" */));
const LivestreamPage = lazyImport(() => import('page/livestream' /* webpackChunkName: "livestream" */)); const LivestreamPage = lazyImport(() => import('page/livestream' /* webpackChunkName: "livestream" */));
const Yrbl = lazyImport(() => import('component/yrbl' /* webpackChunkName: "yrbl" */)); const Yrbl = lazyImport(() => import('component/yrbl' /* webpackChunkName: "yrbl" */));
const isDev = process.env.NODE_ENV !== 'production';
type Props = { type Props = {
isResolvingUri: boolean, isResolvingUri: boolean,
@ -85,7 +88,8 @@ function ShowPage(props: Props) {
const canonicalUrlPath = '/' + canonicalUrl.replace(/^lbry:\/\//, '').replace(/#/g, ':'); const canonicalUrlPath = '/' + canonicalUrl.replace(/^lbry:\/\//, '').replace(/#/g, ':');
// Only redirect if we are in lbry.tv land // Only redirect if we are in lbry.tv land
// replaceState will fail if on a different domain (like webcache.googleusercontent.com) // replaceState will fail if on a different domain (like webcache.googleusercontent.com)
if (canonicalUrlPath !== window.location.pathname && DOMAIN === window.location.hostname) { const hostname = isDev ? 'localhost' : DOMAIN;
if (canonicalUrlPath !== window.location.pathname && hostname === window.location.hostname) {
const urlParams = new URLSearchParams(search); const urlParams = new URLSearchParams(search);
if (urlParams.get(COLLECTIONS_CONSTS.COLLECTION_ID)) { if (urlParams.get(COLLECTIONS_CONSTS.COLLECTION_ID)) {
const listId = urlParams.get(COLLECTIONS_CONSTS.COLLECTION_ID) || ''; const listId = urlParams.get(COLLECTIONS_CONSTS.COLLECTION_ID) || '';

View file

@ -9,18 +9,23 @@ const {
SITE_DESCRIPTION, SITE_DESCRIPTION,
SITE_NAME, SITE_NAME,
FAVICON, FAVICON,
LBRY_WEB_API,
} = require('../../config.js'); } = require('../../config.js');
const { Lbry } = require('lbry-redux');
const { generateEmbedUrl, generateStreamUrl, generateDirectUrl } = require('../../ui/util/web'); const { generateEmbedUrl, generateStreamUrl, generateDirectUrl } = require('../../ui/util/web');
const PAGES = require('../../ui/constants/pages'); const PAGES = require('../../ui/constants/pages');
const { CATEGORY_METADATA } = require('./category-metadata'); const { CATEGORY_METADATA } = require('./category-metadata');
const { getClaim } = require('./chainquery'); const { parseURI, normalizeURI } = require('lbry-redux');
const { parseURI } = require('lbry-redux');
const fs = require('fs'); const fs = require('fs');
const path = require('path'); const path = require('path');
const moment = require('moment'); const moment = require('moment');
const removeMd = require('remove-markdown'); const removeMd = require('remove-markdown');
const { getJsBundleId } = require('../bundle-id.js'); const { getJsBundleId } = require('../bundle-id.js');
const jsBundleId = getJsBundleId(); const jsBundleId = getJsBundleId();
const SDK_API_PATH = `${LBRY_WEB_API}/api/v1`;
const PROXY_URL = `${SDK_API_PATH}/proxy`;
Lbry.setDaemonConnectionString(PROXY_URL);
function insertToHead(fullHtml, htmlToInsert) { function insertToHead(fullHtml, htmlToInsert) {
const beginStr = '<!-- VARIABLE_HEAD_BEGIN -->'; const beginStr = '<!-- VARIABLE_HEAD_BEGIN -->';
@ -40,6 +45,10 @@ function truncateDescription(description) {
return description.length > 200 ? description.substr(0, 200) + '...' : description; return description.length > 200 ? description.substr(0, 200) + '...' : description;
} }
function normalizeClaimUrl(url) {
return normalizeURI(url.replace(/:/g, '#'));
}
function escapeHtmlProperty(property) { function escapeHtmlProperty(property) {
return property return property
? String(property) ? String(property)
@ -122,20 +131,33 @@ function buildBasicOgMetadata() {
function buildClaimOgMetadata(uri, claim, overrideOptions = {}) { function buildClaimOgMetadata(uri, claim, overrideOptions = {}) {
// Initial setup for claim based og metadata // Initial setup for claim based og metadata
const { claimName } = parseURI(uri); const { claimName } = parseURI(uri);
const claimTitle = escapeHtmlProperty(claim.title ? claim.title : claimName); const { meta, value, signing_channel } = claim;
const fee = value && value.fee && (Number(value.fee.amount) || 0);
const tags = value && value.tags;
const media = value && (value.video || value.audio || value.image);
const source = value && value.source;
const channel = signing_channel && signing_channel.name;
const thumbnail = value && value.thumbnail && value.thumbnail.url;
const mediaType = source && source.media_type;
const mediaDuration = media && media.duration;
const claimTitle = escapeHtmlProperty((value && value.title) || claimName);
const releaseTime = (value && value.release_time) || (meta && meta.creation_timestamp) || 0;
const claimDescription = const claimDescription =
claim.description && claim.description.length > 0 value && value.description && value.description.length > 0
? escapeHtmlProperty(truncateDescription(claim.description)) ? escapeHtmlProperty(truncateDescription(value.description))
: `View ${claimTitle} on ${SITE_NAME}`; : `View ${claimTitle} on ${SITE_NAME}`;
const claimLanguage = escapeHtmlProperty(claim.language) || 'en_US';
const claimLanguage =
value && value.languages && value.languages.length > 0 ? escapeHtmlProperty(value.languages[0]) : 'en_US';
let imageThumbnail; let imageThumbnail;
if (Number(claim.fee) <= 0 && claim.source_media_type && claim.source_media_type.startsWith('image/')) { if (fee <= 0 && mediaType && mediaType.startsWith('image/')) {
imageThumbnail = generateStreamUrl(claim.name, claim.claim_id); imageThumbnail = generateStreamUrl(claim.name, claim.claim_id);
} }
const claimThumbnail =
escapeHtmlProperty(claim.thumbnail_url) || imageThumbnail || OG_IMAGE_URL || `${URL}/public/v2-og.png`; const claimThumbnail = escapeHtmlProperty(thumbnail) || imageThumbnail || OG_IMAGE_URL || `${URL}/public/v2-og.png`;
// Allow for ovverriding default claim based og metadata // Allow for ovverriding default claim based og metadata
const title = overrideOptions.title || claimTitle; const title = overrideOptions.title || claimTitle;
@ -148,8 +170,9 @@ function buildClaimOgMetadata(uri, claim, overrideOptions = {}) {
head += '<meta charset="utf8"/>'; head += '<meta charset="utf8"/>';
head += `<title>${title}</title>`; head += `<title>${title}</title>`;
head += `<meta name="description" content="${cleanDescription}"/>`; head += `<meta name="description" content="${cleanDescription}"/>`;
if (claim.tags) {
head += `<meta name="keywords" content="${claim.tags.toString()}"/>`; if (tags && tags.length > 0) {
head += `<meta name="keywords" content="${tags.toString()}"/>`;
} }
head += `<meta name="twitter:image" content="${claimThumbnail}"/>`; head += `<meta name="twitter:image" content="${claimThumbnail}"/>`;
@ -165,33 +188,28 @@ function buildClaimOgMetadata(uri, claim, overrideOptions = {}) {
head += `<meta name="twitter:url" content="${URL}/${claim.name}:${claim.claim_id}"/>`; head += `<meta name="twitter:url" content="${URL}/${claim.name}:${claim.claim_id}"/>`;
head += `<link rel="canonical" content="${SITE_CANONICAL_URL || URL}/${claim.name}:${claim.claim_id}"/>`; head += `<link rel="canonical" content="${SITE_CANONICAL_URL || URL}/${claim.name}:${claim.claim_id}"/>`;
if ( if (mediaType && (mediaType.startsWith('video/') || mediaType.startsWith('audio/'))) {
claim.source_media_type &&
(claim.source_media_type.startsWith('video/') || claim.source_media_type.startsWith('audio/'))
) {
const videoUrl = generateEmbedUrl(claim.name, claim.claim_id); const videoUrl = generateEmbedUrl(claim.name, claim.claim_id);
head += `<meta property="og:video" content="${videoUrl}" />`; head += `<meta property="og:video" content="${videoUrl}" />`;
head += `<meta property="og:video:secure_url" content="${videoUrl}" />`; head += `<meta property="og:video:secure_url" content="${videoUrl}" />`;
head += `<meta property="og:video:type" content="${claim.source_media_type}" />`; head += `<meta property="og:video:type" content="${mediaType}" />`;
if (claim.channel) { if (channel) {
head += `<meta name="og:video:series" content="${claim.channel}"/>`; head += `<meta name="og:video:series" content="${channel}"/>`;
} }
head += `<meta name="twitter:card" content="player"/>`; head += `<meta name="twitter:card" content="player"/>`;
head += `<meta name="twitter:player" content="${videoUrl}" />`; head += `<meta name="twitter:player" content="${videoUrl}" />`;
if (claim.release_time) { if (releaseTime) {
var release = new Date(claim.release_time * 1000).toISOString(); var release = new Date(releaseTime * 1000).toISOString();
head += `<meta property="og:video:release_date" content="${release}"/>`; head += `<meta property="og:video:release_date" content="${release}"/>`;
} }
if (claim.duration) { if (mediaDuration) {
head += `<meta property="og:video:duration" content="${claim.duration}"/>`; head += `<meta property="og:video:duration" content="${mediaDuration}"/>`;
} else if (claim.audio_duration) {
head += `<meta property="og:video:duration" content="${claim.audio_duration}"/>`;
} }
if (claim.frame_width && claim.frame_height) { if (media && media.width && media.height) {
head += `<meta property="og:video:width" content="${claim.frame_width}"/>`; head += `<meta property="og:video:width" content="${media.width}"/>`;
head += `<meta property="og:video:height" content="${claim.frame_height}"/>`; head += `<meta property="og:video:height" content="${media.height}"/>`;
head += `<meta name="twitter:player:width" content="${claim.frame_width}">`; head += `<meta name="twitter:player:width" content="${media.width}">`;
head += `<meta name="twitter:player:height" content="${claim.frame_height}">`; head += `<meta name="twitter:player:height" content="${media.height}">`;
} }
} else { } else {
head += `<meta name="twitter:card" content="summary_large_image"/>`; head += `<meta name="twitter:card" content="summary_large_image"/>`;
@ -200,19 +218,27 @@ function buildClaimOgMetadata(uri, claim, overrideOptions = {}) {
return head; return head;
} }
function buildGoogleVideoMetadata(claimUri, claim) { function buildGoogleVideoMetadata(uri, claim) {
if (!claim.source_media_type || !claim.source_media_type.startsWith('video/')) { const { claimName } = parseURI(uri);
const { meta, value } = claim;
const media = value && value.video;
const source = value && value.source;
const thumbnail = value && value.thumbnail && value.thumbnail.url;
const mediaType = source && source.media_type;
const mediaDuration = media && media.duration;
const claimTitle = escapeHtmlProperty((value && value.title) || claimName);
const releaseTime = (value && value.release_time) || (meta && meta.creation_timestamp) || 0;
const claimDescription =
value && value.description && value.description.length > 0
? escapeHtmlProperty(truncateDescription(value.description))
: `View ${claimTitle} on ${SITE_NAME}`;
if (!mediaType || !mediaType.startsWith('video/')) {
return ''; return '';
} }
const { claimName } = parseURI(claimUri); const claimThumbnail = escapeHtmlProperty(thumbnail) || OG_IMAGE_URL || `${URL}/public/v2-og.png`;
const claimTitle = escapeHtmlProperty(claim.title ? claim.title : claimName);
const claimDescription =
claim.description && claim.description.length > 0
? escapeHtmlProperty(truncateDescription(claim.description))
: `View ${claimTitle} on ${SITE_NAME}`;
const claimThumbnail = escapeHtmlProperty(claim.thumbnail_url) || OG_IMAGE_URL || `${URL}/public/v2-og.png`;
const releaseTime = claim.release_time || 0;
// https://developers.google.com/search/docs/data-types/video // https://developers.google.com/search/docs/data-types/video
const googleVideoMetadata = { const googleVideoMetadata = {
@ -224,7 +250,7 @@ function buildGoogleVideoMetadata(claimUri, claim) {
thumbnailUrl: `${claimThumbnail}`, thumbnailUrl: `${claimThumbnail}`,
uploadDate: `${new Date(releaseTime * 1000).toISOString()}`, uploadDate: `${new Date(releaseTime * 1000).toISOString()}`,
// --- Recommended --- // --- Recommended ---
duration: claim.duration ? moment.duration(claim.duration * 1000).toISOString() : undefined, duration: mediaDuration ? moment.duration(mediaDuration * 1000).toISOString() : undefined,
contentUrl: generateDirectUrl(claim.name, claim.claim_id), contentUrl: generateDirectUrl(claim.name, claim.claim_id),
embedUrl: generateEmbedUrl(claim.name, claim.claim_id), embedUrl: generateEmbedUrl(claim.name, claim.claim_id),
}; };
@ -242,24 +268,20 @@ function buildGoogleVideoMetadata(claimUri, claim) {
); );
} }
async function getClaimFromChainqueryOrRedirect(ctx, url, ignoreRedirect = false) { async function resolveClaimOrRedirect(ctx, url, ignoreRedirect = false) {
const { isChannel, streamName, channelName, channelClaimId, streamClaimId } = parseURI(url); let claim;
const claimName = isChannel ? '@' + channelName : streamName; try {
const claimId = isChannel ? channelClaimId : streamClaimId; const response = await Lbry.resolve({ urls: [url] });
if (response && response[url] && !response[url].error) {
const rows = await getClaim(claimName, claimId, channelName, channelClaimId); claim = response && response[url];
if (rows && rows.length) { const isRepost = claim.reposted_claim && claim.reposted_claim.name && claim.reposted_claim.claim_id;
const claim = rows[0]; if (isRepost && !ignoreRedirect) {
ctx.redirect(`/${claim.reposted_claim.name}:${claim.reposted_claim.claim_id}`);
if (claim.reposted_name && claim.reposted_claim_id && !ignoreRedirect) { return;
ctx.redirect(`/${claim.reposted_name}:${claim.reposted_claim_id}`); }
return;
} }
} catch {}
return claim; return claim;
}
return undefined;
} }
let html; let html;
@ -278,12 +300,10 @@ async function getHtml(ctx) {
const embedPath = `/$/${PAGES.EMBED}/`; const embedPath = `/$/${PAGES.EMBED}/`;
if (requestPath.includes(invitePath)) { if (requestPath.includes(invitePath)) {
const inviteChannel = requestPath.slice(invitePath.length).replace(/:/g, '#');
const inviteChannelUrl = `lbry://${inviteChannel}`;
try { try {
parseURI(inviteChannelUrl); const inviteChannel = requestPath.slice(invitePath.length);
const claim = await getClaimFromChainqueryOrRedirect(ctx, inviteChannelUrl); const inviteChannelUrl = normalizeClaimUrl(inviteChannel);
const claim = await resolveClaimOrRedirect(ctx, inviteChannelUrl);
const invitePageMetadata = buildClaimOgMetadata(inviteChannelUrl, claim, { const invitePageMetadata = buildClaimOgMetadata(inviteChannelUrl, claim, {
title: `Join ${claim.name} on ${SITE_NAME}`, title: `Join ${claim.name} on ${SITE_NAME}`,
description: `Join ${claim.name} on ${SITE_NAME}, a content wonderland owned by everyone (and no one).`, description: `Join ${claim.name} on ${SITE_NAME}, a content wonderland owned by everyone (and no one).`,
@ -302,8 +322,8 @@ async function getHtml(ctx) {
} }
if (requestPath.includes(embedPath)) { if (requestPath.includes(embedPath)) {
const claimUri = requestPath.replace(embedPath, '').replace(/:/g, '#'); const claimUri = normalizeClaimUrl(requestPath.replace(embedPath, ''));
const claim = await getClaimFromChainqueryOrRedirect(ctx, claimUri, true); const claim = await resolveClaimOrRedirect(ctx, claimUri, true);
if (claim) { if (claim) {
const ogMetadata = buildClaimOgMetadata(claimUri, claim); const ogMetadata = buildClaimOgMetadata(claimUri, claim);
@ -325,8 +345,8 @@ async function getHtml(ctx) {
} }
if (!requestPath.includes('$')) { if (!requestPath.includes('$')) {
const claimUri = requestPath.slice(1).replace(/:/g, '#'); const claimUri = normalizeClaimUrl(requestPath.slice(1));
const claim = await getClaimFromChainqueryOrRedirect(ctx, claimUri); const claim = await resolveClaimOrRedirect(ctx, claimUri);
if (claim) { if (claim) {
const ogMetadata = buildClaimOgMetadata(claimUri, claim); const ogMetadata = buildClaimOgMetadata(claimUri, claim);