Tips for retrieving specific information from Wikipedia using AJAX

Question

Tips for retrieving specific information from Wikipedia using AJAX

Is there a way to retrieve the information consistently displayed in the right box during searches using AJAX? I've tried using the Wikipedia API, but haven't been able to find the specific information I need.

https://i.sstatic.net/wqJEc.png

javascript json ajax api wikipedia

Answer 1

Answer №1

Oh my goodness, I never imagined spending so much time responding to this query on stackoverflow,

Presented below is a crude yet functional code snippet:

// wikipedia article (in url)
const wiki_article_title = 'Grand_Theft_Auto_V';

// please check https://www.mediawiki.org/wiki/API:Get_the_contents_of_a_page
const url_api = `https://en.wikipedia.org/w/api.php?action=parse&page=${wiki_article_title}&prop=text&formatversion=2&origin=*`;

function extractInfoboxFromWiki(doc) {
  // here we extract the json provided by api
  const json = doc.querySelector('pre');
  const obj = JSON.parse(json.innerText);
  let html = obj.parse.text;

  // for whatever reason '\n' substring are present in html text
  // so we remove them with a regex to not break 'JSON.parse()'
  html = html.replace(/\\n/gm, '');

  // get the interesting part of api reponse
  const node = document.createElement('div');
  node.innerHTML = html;
  const infobox = node.querySelector('.infobox');
  let infos = [...infobox.firstChild.children];

  let output = {};

  // parse title
  output['title'] = infos[0].querySelector('th').innerText;
  infos.shift();

  // parse image url
  output['image_url'] = infos[0].querySelector('a').getAttribute("href");
  infos.shift();

  // traverse the nodes to map captions with values
  infos.forEach( tr => {
    const key = tr.querySelector('th').innerText;

    if(tr.querySelector('ul')) {
      const lis = tr.querySelectorAll('li');
      const values = [...lis].map( li => li.innerText);
      output[key] = values;
    } else {
      const value = tr.querySelector('td').innerText;
      output[key] = value;
    }

  });

  // return beautified json
  return JSON.stringify(output, null, 4);
}

fetch(url_api)
  .then(response => response.text())
  .then(text => {
    const parser = new DOMParser();
    const doc = parser.parseFromString(text, 'text/html');

    const DESIRED_RESULT = extractInfoboxFromWiki(doc);
    const formattedOutput = `<pre>${DESIRED_RESULT}</pre>`;

    document.write(formattedOutput);
  });

If you test it with the Grand Theft Auto V article, you will observe:

{
    "title": "Grand Theft Auto V",
    "image_url": "/wiki/File:Grand_Theft_Auto_V.png",
    "Developer(s)": "Rockstar North[a]",
    "Publisher(s)": "Rockstar Games",
    "Producer(s)": [
        "Leslie Benzies",
        "Imran Sarwar"
    ],
    "Designer(s)": [
        "Leslie Benzies",
        "Imran Sarwar"
    ],
    "Programmer(s)": "Adam Fowler",
    "Artist(s)": "Aaron Garbut",
    "Writer(s)": [
        "Dan Houser",
        "Rupert Humphries",
        "Michael Unsworth"
    ],
    "Composer(s)": [
        "Tangerine Dream",
        "Woody Jackson",
        "The Alchemist",
        "Oh No"
    ],
    "Series": "Grand Theft Auto",
    "Engine": "RAGE",
    "Platform(s)": [
        "PlayStation 3",
        "Xbox 360",
        "PlayStation 4&q...</answer1>
<exanswer1><div class="answer accepted" i="66002290" l="4.0" c="1612216816" m="1612229346" v="1" a="U2ltb24gRGVoYXV0" ai="12153710">
<p>ok, omg, I never spend this much time for answering a question on stackoverflow,</p>
<p>so you have a working snippet below, it's dirty but it's working :)</p>
<p><div>
<div>
<pre class="lang-js"><code>// wikipedia article (in url)
const wiki_article_title = 'Grand_Theft_Auto_V';

// please check https://www.mediawiki.org/wiki/API:Get_the_contents_of_a_page
const url_api = `https://en.wikipedia.org/w/api.php?action=parse&page=${wiki_article_title}&prop=text&formatversion=2&origin=*`;

function extractInfoboxFromWiki(doc) {
  // here we extract the json provided by api
  const json = doc.querySelector('pre');
  const obj = JSON.parse(json.innerText);
  let html = obj.parse.text;

  // for whatever reason '\n' substring are present in html text
  // so we remove them with a regex to not break 'JSON.parse()'
  html = html.replace(/\\n/gm, '');

  // get the interesting part of api reponse
  const node = document.createElement('div');
  node.innerHTML = html;
  const infobox = node.querySelector('.infobox');
  let infos = [...infobox.firstChild.children];

  let output = {};

  // parse title
  output['title'] = infos[0].querySelector('th').innerText;
  infos.shift();

  // parse image url
  output['image_url'] = infos[0].querySelector('a').getAttribute("href");
  infos.shift();

  // traverse the nodes to map captions with values
  infos.forEach( tr => {
    const key = tr.querySelector('th').innerText;

    if(tr.querySelector('ul')) {
      const lis = tr.querySelectorAll('li');
      const values = [...lis].map( li => li.innerText);
      output[key] = values;
    } else {
      const value = tr.querySelector('td').innerText;
      output[key] = value;
    }

  });

  // return beautified json
  return JSON.stringify(output, null, 4);
}

fetch(url_api)
  .then(response => response.text())
  .then(text => {
    const parser = new DOMParser();
    const doc = parser.parseFromString(text, 'text/html');

    const WHAT_YOU_WANT = extractInfoboxFromWiki(doc);
    const formated = `<pre>${WHAT_YOU_WANT}</pre>`;

    document.write(formated);
  });

Answer 2

Oh my goodness, I never imagined spending so much time responding to this query on stackoverflow,

Presented below is a crude yet functional code snippet:

// wikipedia article (in url)
const wiki_article_title = 'Grand_Theft_Auto_V';

// please check https://www.mediawiki.org/wiki/API:Get_the_contents_of_a_page
const url_api = `https://en.wikipedia.org/w/api.php?action=parse&page=${wiki_article_title}&prop=text&formatversion=2&origin=*`;

function extractInfoboxFromWiki(doc) {
  // here we extract the json provided by api
  const json = doc.querySelector('pre');
  const obj = JSON.parse(json.innerText);
  let html = obj.parse.text;

  // for whatever reason '\n' substring are present in html text
  // so we remove them with a regex to not break 'JSON.parse()'
  html = html.replace(/\\n/gm, '');

  // get the interesting part of api reponse
  const node = document.createElement('div');
  node.innerHTML = html;
  const infobox = node.querySelector('.infobox');
  let infos = [...infobox.firstChild.children];

  let output = {};

  // parse title
  output['title'] = infos[0].querySelector('th').innerText;
  infos.shift();

  // parse image url
  output['image_url'] = infos[0].querySelector('a').getAttribute("href");
  infos.shift();

  // traverse the nodes to map captions with values
  infos.forEach( tr => {
    const key = tr.querySelector('th').innerText;

    if(tr.querySelector('ul')) {
      const lis = tr.querySelectorAll('li');
      const values = [...lis].map( li => li.innerText);
      output[key] = values;
    } else {
      const value = tr.querySelector('td').innerText;
      output[key] = value;
    }

  });

  // return beautified json
  return JSON.stringify(output, null, 4);
}

fetch(url_api)
  .then(response => response.text())
  .then(text => {
    const parser = new DOMParser();
    const doc = parser.parseFromString(text, 'text/html');

    const DESIRED_RESULT = extractInfoboxFromWiki(doc);
    const formattedOutput = `<pre>${DESIRED_RESULT}</pre>`;

    document.write(formattedOutput);
  });

If you test it with the Grand Theft Auto V article, you will observe:

{
    "title": "Grand Theft Auto V",
    "image_url": "/wiki/File:Grand_Theft_Auto_V.png",
    "Developer(s)": "Rockstar North[a]",
    "Publisher(s)": "Rockstar Games",
    "Producer(s)": [
        "Leslie Benzies",
        "Imran Sarwar"
    ],
    "Designer(s)": [
        "Leslie Benzies",
        "Imran Sarwar"
    ],
    "Programmer(s)": "Adam Fowler",
    "Artist(s)": "Aaron Garbut",
    "Writer(s)": [
        "Dan Houser",
        "Rupert Humphries",
        "Michael Unsworth"
    ],
    "Composer(s)": [
        "Tangerine Dream",
        "Woody Jackson",
        "The Alchemist",
        "Oh No"
    ],
    "Series": "Grand Theft Auto",
    "Engine": "RAGE",
    "Platform(s)": [
        "PlayStation 3",
        "Xbox 360",
        "PlayStation 4&q...</answer1>
<exanswer1><div class="answer accepted" i="66002290" l="4.0" c="1612216816" m="1612229346" v="1" a="U2ltb24gRGVoYXV0" ai="12153710">
<p>ok, omg, I never spend this much time for answering a question on stackoverflow,</p>
<p>so you have a working snippet below, it's dirty but it's working :)</p>
<p><div>
<div>
<pre class="lang-js"><code>// wikipedia article (in url)
const wiki_article_title = 'Grand_Theft_Auto_V';

// please check https://www.mediawiki.org/wiki/API:Get_the_contents_of_a_page
const url_api = `https://en.wikipedia.org/w/api.php?action=parse&page=${wiki_article_title}&prop=text&formatversion=2&origin=*`;

function extractInfoboxFromWiki(doc) {
  // here we extract the json provided by api
  const json = doc.querySelector('pre');
  const obj = JSON.parse(json.innerText);
  let html = obj.parse.text;

  // for whatever reason '\n' substring are present in html text
  // so we remove them with a regex to not break 'JSON.parse()'
  html = html.replace(/\\n/gm, '');

  // get the interesting part of api reponse
  const node = document.createElement('div');
  node.innerHTML = html;
  const infobox = node.querySelector('.infobox');
  let infos = [...infobox.firstChild.children];

  let output = {};

  // parse title
  output['title'] = infos[0].querySelector('th').innerText;
  infos.shift();

  // parse image url
  output['image_url'] = infos[0].querySelector('a').getAttribute("href");
  infos.shift();

  // traverse the nodes to map captions with values
  infos.forEach( tr => {
    const key = tr.querySelector('th').innerText;

    if(tr.querySelector('ul')) {
      const lis = tr.querySelectorAll('li');
      const values = [...lis].map( li => li.innerText);
      output[key] = values;
    } else {
      const value = tr.querySelector('td').innerText;
      output[key] = value;
    }

  });

  // return beautified json
  return JSON.stringify(output, null, 4);
}

fetch(url_api)
  .then(response => response.text())
  .then(text => {
    const parser = new DOMParser();
    const doc = parser.parseFromString(text, 'text/html');

    const WHAT_YOU_WANT = extractInfoboxFromWiki(doc);
    const formated = `<pre>${WHAT_YOU_WANT}</pre>`;

    document.write(formated);
  });

Tips for retrieving specific information from Wikipedia using AJAX

Answer №1

Similar questions

Utilize CSS to vertically align buttons

The Ajax submission is functioning perfectly with the exception of the notification feature

Encountering difficulty in retrieving the outcome of the initial HTTP request while utilizing the switchMap function in RxJS

How can I add a blank selection at the bottom of my ng-options dropdown?

The deprecated body parser is throwing an error due to the lack of the extended option

What are some ways to utilize an empty array that has been declared in React's initial state?

Calculating the mean value of the numbers provided in the input

What is the optimal method for presenting data on a device from a massive database?

multer - the file uploaded by the request is not defined

I need help figuring out how to mention an id using a concatenated variable in the jquery appendTo() method

Load an iframe with Ajax without the need to refresh the page

Developing a MySQL DB-driven Node.js dashboard without the need for a poller

Jquery form submission error occurring with file upload functionality

Retrieve a specific item from a JSON response using Node.js

Why is it necessary to type in localhost in the address bar when using node.js and socket.io instead of simply opening the client.html file directly?

Tips on effectively managing an XMLhttp request to a PHP controller

The issue of Facebook's AJAX system dropping user sessions

Unable to save data retrieved using jQuery JSONP

Issue with concatenation function in JOLT causing errors

What is Mozilla's reasoning for deeming Conditional catch-blocks as non-conventional?