Tips for retrieving specific information from Wikipedia using AJAX

Question

Tips for retrieving specific information from Wikipedia using AJAX

Is there a way to retrieve the information consistently displayed in the right box during searches using AJAX? I've tried using the Wikipedia API, but haven't been able to find the specific information I need.

javascript json ajax api wikipedia

Answer 1

Answer №1

Oh my goodness, I never imagined spending so much time responding to this query on stackoverflow,

Presented below is a crude yet functional code snippet:

// wikipedia article (in url)
const wiki_article_title = 'Grand_Theft_Auto_V';

// please check https://www.mediawiki.org/wiki/API:Get_the_contents_of_a_page
const url_api = `https://en.wikipedia.org/w/api.php?action=parse&page=${wiki_article_title}&prop=text&formatversion=2&origin=*`;

function extractInfoboxFromWiki(doc) {
  // here we extract the json provided by api
  const json = doc.querySelector('pre');
  const obj = JSON.parse(json.innerText);
  let html = obj.parse.text;

  // for whatever reason '\n' substring are present in html text
  // so we remove them with a regex to not break 'JSON.parse()'
  html = html.replace(/\\n/gm, '');

  // get the interesting part of api reponse
  const node = document.createElement('div');
  node.innerHTML = html;
  const infobox = node.querySelector('.infobox');
  let infos = [...infobox.firstChild.children];

  let output = {};

  // parse title
  output['title'] = infos[0].querySelector('th').innerText;
  infos.shift();

  // parse image url
  output['image_url'] = infos[0].querySelector('a').getAttribute("href");
  infos.shift();

  // traverse the nodes to map captions with values
  infos.forEach( tr => {
    const key = tr.querySelector('th').innerText;

    if(tr.querySelector('ul')) {
      const lis = tr.querySelectorAll('li');
      const values = [...lis].map( li => li.innerText);
      output[key] = values;
    } else {
      const value = tr.querySelector('td').innerText;
      output[key] = value;
    }

  });

  // return beautified json
  return JSON.stringify(output, null, 4);
}

fetch(url_api)
  .then(response => response.text())
  .then(text => {
    const parser = new DOMParser();
    const doc = parser.parseFromString(text, 'text/html');

    const DESIRED_RESULT = extractInfoboxFromWiki(doc);
    const formattedOutput = `<pre>${DESIRED_RESULT}</pre>`;

    document.write(formattedOutput);
  });

If you test it with the Grand Theft Auto V article, you will observe:

{
    "title": "Grand Theft Auto V",
    "image_url": "/wiki/File:Grand_Theft_Auto_V.png",
    "Developer(s)": "Rockstar North[a]",
    "Publisher(s)": "Rockstar Games",
    "Producer(s)": [
        "Leslie Benzies",
        "Imran Sarwar"
    ],
    "Designer(s)": [
        "Leslie Benzies",
        "Imran Sarwar"
    ],
    "Programmer(s)": "Adam Fowler",
    "Artist(s)": "Aaron Garbut",
    "Writer(s)": [
        "Dan Houser",
        "Rupert Humphries",
        "Michael Unsworth"
    ],
    "Composer(s)": [
        "Tangerine Dream",
        "Woody Jackson",
        "The Alchemist",
        "Oh No"
    ],
    "Series": "Grand Theft Auto",
    "Engine": "RAGE",
    "Platform(s)": [
        "PlayStation 3",
        "Xbox 360",
        "PlayStation 4&q...</answer1>
<exanswer1><div class="answer accepted" i="66002290" l="4.0" c="1612216816" m="1612229346" v="1" a="U2ltb24gRGVoYXV0" ai="12153710">
<p>ok, omg, I never spend this much time for answering a question on stackoverflow,</p>
<p>so you have a working snippet below, it's dirty but it's working :)</p>
<p><div>
<div>
<pre class="lang-js"><code>// wikipedia article (in url)
const wiki_article_title = 'Grand_Theft_Auto_V';

// please check https://www.mediawiki.org/wiki/API:Get_the_contents_of_a_page
const url_api = `https://en.wikipedia.org/w/api.php?action=parse&page=${wiki_article_title}&prop=text&formatversion=2&origin=*`;

function extractInfoboxFromWiki(doc) {
  // here we extract the json provided by api
  const json = doc.querySelector('pre');
  const obj = JSON.parse(json.innerText);
  let html = obj.parse.text;

  // for whatever reason '\n' substring are present in html text
  // so we remove them with a regex to not break 'JSON.parse()'
  html = html.replace(/\\n/gm, '');

  // get the interesting part of api reponse
  const node = document.createElement('div');
  node.innerHTML = html;
  const infobox = node.querySelector('.infobox');
  let infos = [...infobox.firstChild.children];

  let output = {};

  // parse title
  output['title'] = infos[0].querySelector('th').innerText;
  infos.shift();

  // parse image url
  output['image_url'] = infos[0].querySelector('a').getAttribute("href");
  infos.shift();

  // traverse the nodes to map captions with values
  infos.forEach( tr => {
    const key = tr.querySelector('th').innerText;

    if(tr.querySelector('ul')) {
      const lis = tr.querySelectorAll('li');
      const values = [...lis].map( li => li.innerText);
      output[key] = values;
    } else {
      const value = tr.querySelector('td').innerText;
      output[key] = value;
    }

  });

  // return beautified json
  return JSON.stringify(output, null, 4);
}

fetch(url_api)
  .then(response => response.text())
  .then(text => {
    const parser = new DOMParser();
    const doc = parser.parseFromString(text, 'text/html');

    const WHAT_YOU_WANT = extractInfoboxFromWiki(doc);
    const formated = `<pre>${WHAT_YOU_WANT}</pre>`;

    document.write(formated);
  });

Answer 2

Oh my goodness, I never imagined spending so much time responding to this query on stackoverflow,

Presented below is a crude yet functional code snippet:

// wikipedia article (in url)
const wiki_article_title = 'Grand_Theft_Auto_V';

// please check https://www.mediawiki.org/wiki/API:Get_the_contents_of_a_page
const url_api = `https://en.wikipedia.org/w/api.php?action=parse&page=${wiki_article_title}&prop=text&formatversion=2&origin=*`;

function extractInfoboxFromWiki(doc) {
  // here we extract the json provided by api
  const json = doc.querySelector('pre');
  const obj = JSON.parse(json.innerText);
  let html = obj.parse.text;

  // for whatever reason '\n' substring are present in html text
  // so we remove them with a regex to not break 'JSON.parse()'
  html = html.replace(/\\n/gm, '');

  // get the interesting part of api reponse
  const node = document.createElement('div');
  node.innerHTML = html;
  const infobox = node.querySelector('.infobox');
  let infos = [...infobox.firstChild.children];

  let output = {};

  // parse title
  output['title'] = infos[0].querySelector('th').innerText;
  infos.shift();

  // parse image url
  output['image_url'] = infos[0].querySelector('a').getAttribute("href");
  infos.shift();

  // traverse the nodes to map captions with values
  infos.forEach( tr => {
    const key = tr.querySelector('th').innerText;

    if(tr.querySelector('ul')) {
      const lis = tr.querySelectorAll('li');
      const values = [...lis].map( li => li.innerText);
      output[key] = values;
    } else {
      const value = tr.querySelector('td').innerText;
      output[key] = value;
    }

  });

  // return beautified json
  return JSON.stringify(output, null, 4);
}

fetch(url_api)
  .then(response => response.text())
  .then(text => {
    const parser = new DOMParser();
    const doc = parser.parseFromString(text, 'text/html');

    const DESIRED_RESULT = extractInfoboxFromWiki(doc);
    const formattedOutput = `<pre>${DESIRED_RESULT}</pre>`;

    document.write(formattedOutput);
  });

If you test it with the Grand Theft Auto V article, you will observe:

{
    "title": "Grand Theft Auto V",
    "image_url": "/wiki/File:Grand_Theft_Auto_V.png",
    "Developer(s)": "Rockstar North[a]",
    "Publisher(s)": "Rockstar Games",
    "Producer(s)": [
        "Leslie Benzies",
        "Imran Sarwar"
    ],
    "Designer(s)": [
        "Leslie Benzies",
        "Imran Sarwar"
    ],
    "Programmer(s)": "Adam Fowler",
    "Artist(s)": "Aaron Garbut",
    "Writer(s)": [
        "Dan Houser",
        "Rupert Humphries",
        "Michael Unsworth"
    ],
    "Composer(s)": [
        "Tangerine Dream",
        "Woody Jackson",
        "The Alchemist",
        "Oh No"
    ],
    "Series": "Grand Theft Auto",
    "Engine": "RAGE",
    "Platform(s)": [
        "PlayStation 3",
        "Xbox 360",
        "PlayStation 4&q...</answer1>
<exanswer1><div class="answer accepted" i="66002290" l="4.0" c="1612216816" m="1612229346" v="1" a="U2ltb24gRGVoYXV0" ai="12153710">
<p>ok, omg, I never spend this much time for answering a question on stackoverflow,</p>
<p>so you have a working snippet below, it's dirty but it's working :)</p>
<p><div>
<div>
<pre class="lang-js"><code>// wikipedia article (in url)
const wiki_article_title = 'Grand_Theft_Auto_V';

// please check https://www.mediawiki.org/wiki/API:Get_the_contents_of_a_page
const url_api = `https://en.wikipedia.org/w/api.php?action=parse&page=${wiki_article_title}&prop=text&formatversion=2&origin=*`;

function extractInfoboxFromWiki(doc) {
  // here we extract the json provided by api
  const json = doc.querySelector('pre');
  const obj = JSON.parse(json.innerText);
  let html = obj.parse.text;

  // for whatever reason '\n' substring are present in html text
  // so we remove them with a regex to not break 'JSON.parse()'
  html = html.replace(/\\n/gm, '');

  // get the interesting part of api reponse
  const node = document.createElement('div');
  node.innerHTML = html;
  const infobox = node.querySelector('.infobox');
  let infos = [...infobox.firstChild.children];

  let output = {};

  // parse title
  output['title'] = infos[0].querySelector('th').innerText;
  infos.shift();

  // parse image url
  output['image_url'] = infos[0].querySelector('a').getAttribute("href");
  infos.shift();

  // traverse the nodes to map captions with values
  infos.forEach( tr => {
    const key = tr.querySelector('th').innerText;

    if(tr.querySelector('ul')) {
      const lis = tr.querySelectorAll('li');
      const values = [...lis].map( li => li.innerText);
      output[key] = values;
    } else {
      const value = tr.querySelector('td').innerText;
      output[key] = value;
    }

  });

  // return beautified json
  return JSON.stringify(output, null, 4);
}

fetch(url_api)
  .then(response => response.text())
  .then(text => {
    const parser = new DOMParser();
    const doc = parser.parseFromString(text, 'text/html');

    const WHAT_YOU_WANT = extractInfoboxFromWiki(doc);
    const formated = `<pre>${WHAT_YOU_WANT}</pre>`;

    document.write(formated);
  });

Tips for retrieving specific information from Wikipedia using AJAX

Answer №1

Similar questions

Use jQuery to remove an ID from an element using AJAX to interact with a database, then update

Tips on securely saving passwords using Greasemonkey

Aligning a div vertically in the center of its parent container

Developing a Typescript module, the dependent module is searching for an import within the local directory but encounters an issue - the module cannot be found and

Utilizing the Google Translate API within an ASP MVC framework to translate a div's content from English to Arabic

Automatically reloading POST request when browser back button is pressed

In Three.js, FBX bones exhibit smooth rotation, but GLTF bones display strange rotation behavior

Learn how to access nested JSON data within a DStream using PySpark

What is the best way to pause the display of dynamic search outcomes in React?

Using Joomla to generate JSON data for AJAX requests

Transform the outcome of Request() into a variable

A method to deactivate a button cell after it has been clicked within a for loop

Having trouble accessing undefined properties? Facing issues with the latest Angular version?

Receiving a null response from an Ajax post request

The term "Cardlist" has not been defined and is therefore causing an

Is it possible to include a visible comment in an ajax call that can be viewed in Fiddler when analyzing the outgoing data?

Switch up the like button for users who have previously liked a post

Tips for limiting a website to load exclusively within an iframe

NodeJS is facing a severe challenge in properly rendering HTML and its accompanying CSS code, causing a major

A guide on incorporating recursion to nest dictionaries within current data records