What steps should be taken in order to ensure the effectiveness of this keyword search

Currently, I am attempting to implement a keyword search feature for my meteor web app. Although the functionality is there, it's running quite slow. The way it works now is that when a user creates an article, they assign keywords to it. The keyS function then queries one article at a time from MongoDB based on a keyword in the search array (skeywords), generates a score for each article, and finally sends the top 100 scored articles back to the user. Is there a more efficient way to query all relevant articles simultaneously?

P.S. Is my approach completely wrong?

This is how the data sent from the client appears:

var keyw = ['java','code','jdk','food','good','cook'];
Meteor.call('keyS', keyw);

The output of 'keyS' is an array of article IDs.

Example:

Sarticles = [someid, someid]

Server-side:

Meteor.methods({
    keyS: function(skeywords) {
        article: 'tempid',
            var score = {
        totalScore: 0
        };
        var potentials = [];
        var badArticles = [];
        var i = 0;
        while (i < skeywords.length) {
            var key = [];
            key.push(skeywords[i]);
            console.log(key);
            if (typeof badarticles == "undefined") {
                var theArticle = Articles.findOne({
                    articlekeywords: {
                        $in: key
                    }
                });
            } else {
                var theArticle = Articles.findOne({
                    $and: [{
                        articlekeywords: {
                            $in: key
                        }
                    }, {
                        _id: {
                            $nin: badArticles
                        }
                    }]
                });
            };
            if (typeof theArticle == "undefined") {
                console.log("no more articles with that keyword")
                i++;
                continue
            }
            score.post = theArticle._id;
            console.log(score.article);
            score.totalScore = 0;
            var points = 0;
            var theKeywords = thearticle.keywords;
            console.log("score worked");
            var points = 0;
            for (var a = 0; a < skeywords.length; a++) {
                var keynumber = theKeywords.indexOf(skeywords[a]);
                if (keynumber > -1) {
                    points++
                } else {
                    continue
                }

            };


            score.totalScore = points;
            console.log(score.totalScore);
            if (score.totalScore > 2) {
            //limiter on number of posts looked at and number added to potentials
                potentials.push({
                    iD: score.post,
                    totalScore: score.totalScore
                });
                var ID = score.article;
                badposts.push(score.article);
                console.log("added to potential" + ID + "to bad");
            } else {
                var badId = score.post;
                console.log("marked as bad" + badId);
                badposts.push(score.post);
            }
        };
        potentials.sort(function(a, b) {
            return b.totalScore - a.totalScore
        })
        for (var b = 0; b < 100; b++) {
            if (typeof potentials[b] == "undefined") {
                break
            };
            var ID = potentials[b].iD;
            Meteor.users.update({
                "_id": this.userId
            }, {
                "$addToSet": {
                    "Sarticles": ID
                }
            });
        }
    }

});

Answer №1

It seems like the issue lies in the server round-trip process. To enhance user experience, it is advisable to implement a publish/subscribe system for the keywords list. This would involve making the list accessible on the client side for searching purposes.

Considering that the keyword list could potentially become very lengthy, my search package (Spomet, still in development) only publishes the 1000 most frequently used words, excluding common stop words such as 'and'.

While my code may not be impeccably organized, it could still be beneficial. Below are links to the client and server-side handling. The client performs the initial search locally before updating with accurate results from the server:

Client-side handling: https://github.com/Crenshinibon/spomet-pkg/blob/master/client.coffee

Server-side code: https://github.com/Crenshinibon/spomet-pkg/blob/master/server.coffee

Another suggestion is to reconsider how you represent your data for keywords. Utilize the keywords as reference points in a separate collection, linking them to arrays of article IDs where they are featured. Research 'Inverted Index' on Wikipedia for further insight into this concept.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Transferring an object from Javascript to C++/CX following a C++/CX interface in Windows Runtime Components

As a newcomer to Windows Runtime Component, I've been exploring ways to accomplish the following task. I'm looking to extend a C++ interface in JavaScript. namespace MySDK { public interface class LoggerPlugin { public: virt ...

Leverage JavaScript to add the rel=0 attribute to a specific URL

Currently, I am integrating videos into my WordPress website using a plugin and I'm looking for a way to ensure that all the YouTube URLs contain rel=0 to eliminate related videos at the end. Can anyone suggest the best approach to achieve this? Any ...

The challenge of loading multiple objects asynchronously in three.js

When I try to load multiple models onto the same scene simultaneously, only one model is successfully loaded. For instance, if I have a building scene with various objects like chairs and toys inside, only one object gets loaded when I try to load them all ...

Tips for adding a scroll-to-top button on a page with only a vertical scroll bar

How can I make the scroll to top/bottom button only show up on pages with a vertical scrollbar? Currently, the button is not appearing when the page contains a vertical scrollbar. Any suggestions on how to implement this feature? ScrollToTopBottom.vue < ...

Generate fresh input fields with distinct identifiers using JavaScript

My challenge is to dynamically create new empty text boxes in JavaScript, each with a unique name, while retaining the text entered in the previous box. I struggled with this for a while and eventually resorted to using PHP, but this resulted in unnecessar ...

Utilizing PHP and Ajax for paginating JSON responses

I have successfully parsed some JSON data from the YouTube API, but I am running into a limitation where only 50 results can be shown per request. I am looking for help on how to implement pagination using either JavaScript or Ajax in my PHP script. The go ...

Optimizing CSS With jQuery During Browser Resize

I am currently facing an issue with recalculating the height of the li element during window resizing or scrolling. Strangely, on page load, the height is no longer being re-calculated and set to the ul's child height. Here is the code I have written ...

Parsing JSON data using multilevel iteration in JavaScript is a complex yet

{ "id":0, "item":[ { "id":"0-", "text":"BlueWing", "userdata":[ { "name":"cid", "content":"10377" } ], "item":[ { "id" ...

Can one transition from using a callback to a returning Promise in order to implement an ErrorFirstCallback strategy?

In Node.js documentation, it is explained that an ErrorFirstCallback is triggered when the referred function fails. Error-first-callbacks in Node.js I have been practicing with this callback pattern and I am curious to know if it is possible to refactor i ...

Troubleshooting problem in Java related to encoding with XMLHttpRequest

Currently, I am utilizing XMLHttpRequest for an ajax call to my server. Let's consider the call: http = new XMLHTTPRequest(); var url = "http://app:8080/search.action?value=ñ" http.open("GET",url,true); http.setRequestHeader("Content-type", "applica ...

Combining duplicate objects in a JavaScript array

I am looking to identify duplicates by country code and merge them into a single object The array structure is: [{...},{...}] Objects {Country_Code: "RU", Country: "Russia", Provider: "Shell1", Price: "0.123"}, {Country_Code: "EN", Country: "Russia", P ...

Sending information from one page to another and then sending it once more

I am currently utilizing the following code to submit a form's data to : <script type="text/javascript"> $(document).ready(function(){ $("#textnextofkin").validate({ debug: false, rules: { ...

The event "click .className" within the Marionette module is failing to trigger

I'm facing an issue with some code that I've written. Here's a snippet of what it looks like: myApp.module(args, function(args){ Views.MainView = Marionette.ItemView.extend({ //template, tagName, className down: false, events: ...

Searching through an array of objects in MongoDB using filters

I have a scenario in my database where I need to extract specific records. Specifically, I am looking to retrieve all records with Type = 1, customerID = "Customer 1", and sort them by transDateTime in ascending order. Can anyone guide me on how to achie ...

Obtain data attributes using JQuery's click event handler

I'm facing an issue with a div structure setup as follows: <div class='bar'> <div class='contents'> <div class='element' data-big='join'>JOIN ME</div> <div class=& ...

What to do when VueJs computed method throws an error: "Cannot access property 'words' of undefined"?

Here is some pseudo code that I've written: var vm = new Vue({ el: '#test', data: { words: [{name:'1'}, {name:'2'}, {name:'3'},{name:'4'},{name:'5'},{name:'6'}], } ...

Securing Your MongoDB Database: Best Practices for Data Encryption

I have a current NodeJS application that allows individuals to save personal details in a MongoDB database. It is crucial for this information to be encrypted or not directly linked to any specific user. Initially, I considered using the NPM mongoose-encr ...

Passing data in Angular 4 with eventEmitter across multiple layers of components

Struggling with a challenge in Angular and need some guidance. I am currently working with Angular 4 and here is the scenario: The app.component.html file contains a wrapper div that I want to be able to change its color by adding a class to it. However ...

Passing parameters to an Angular 2 component

When it comes to passing a string parameter to my component, I need the flexibility to adjust the parameters of services based on the passed value. Here's how I handle it: In my index.html, I simply call my component and pass the required parameter. ...

Utilizing jQuery to correspond with CSS media queries

Continuing from my previous question about an automatic jQuery image slider, you can refer to it here. I have made some changes in my CSS using media queries and I am trying to incorporate them into my JavaScript code using an 'if / else if' sta ...