Movatterモバイル変換


[0]ホーム

URL:


Skip to content
DEV Community
Log in Create account

DEV Community

Cover image for Remove Duplicates With Set() - Full-Stop
Adam Nathaniel Davis
Adam Nathaniel Davis

Posted on • Edited on

     

Remove Duplicates With Set() - Full-Stop

This will probably seem like a trivial and overly simplistic post. But I've seen thewrong answer posted so frequently - even right here on Dev.to - that it's honestly starting to get me kinda annoyed.

Everyone who wants to throw their hat in the ring as some kinda JS "mentor" decides to crank out an article about de-duping arrays. On the surface, it kinda makes sense. Because de-duping an array is a common coding task given to people during whiteboard interviews.

It's one of those questions that seems downright silly for a seasoned dev. But more junior types might struggle with it. More importantly, there are (theoretically)many different ways that you could attack the problem. Thus, it could potentially give the evaluator a great chance to "see" how you think.

So this should be a great chance to write up an awesome how-to article, right? You can display your epic dev knowledge and maybe help some noobs while you burnish your reputation...right??

Unfortunately, I see sooooo many articles where the proposed approach is just downrightwrong. I'm not claiming that other proposed solutions don'twork. I'm saying that other proposed solutions "work" - but they're still very, verywrong.



Alt Text

Use Set() - It'sThat Simple

Let's have a long, detailed, crazy-intense discussion about all the various ways we could de-dupe an array. Or... let's not. That doesn't sound like much fun at all. Instead, let's just useSet() - and then go on to something far more productive.

I've lost track of how many times I've seen - on this site, on Stack Overflow, on Medium, on... wherever - someone claiming to show you how to de-dupe an array. And then they start pulling out a ridiculous mess of.reduce(),.filter(),.map(),.forEach(), or.whateverArrayPrototypeInterestsMeToday().

Stop. It.

Right now. Just...stop it.

Here's your dead-simple rule for de-duping JS arrays:

If the task isonly to de-dupe a JS array containing scalar values, you should be reaching forSet(). Every. Single. Time.


Honestly, this is a fairly narrow use-case in real-life code. I rarely find myself with an array for which Ionly need to de-dupe it. More often than not, there's some combination of tricks I need to perform on the array. And in those cases, it's perfectly natural (and correct) to start reaching for all of thoseArray.prototype functions. But if you truly just need to de-dupe an array then, for the love of god, man, just useSet(). Period.


Alt Text

Secret Knowledge(???)

This reallyshouldn't be any kind of "secret knowledge".Set() isn't a hidden "trick". But most devs rarely useSet() and know little about it. The fact is that I rarely use it myself. But I know what it does. Specifically, I know that it has three very cool features:

  1. Each element in a set must be unique. To be more specific, aSet() will notallow you to create duplicate elements.

  2. It can be initiated with an array - and if that array contains duplicates, the duplicates will be ignored.

  3. You can use the spread operator to convert it back into a plain-ol' array.

When you combine all these features, you have a powerful (and dead-simple) de-duping tool. It looks like this:

consttheDupes=[6,7,8,6,7,2,5,6,7,8,3,6,8,7,2,5];constnoDupes=[...newSet(theDupes)];// [6,7,8,2,5,3]
Enter fullscreen modeExit fullscreen mode

Maybe you're thinking that this isn't often recommended due toperformance?? Umm... no. This technique typically beats every other approach. In fact, the comparison is rarely even close.

De-duping withSet() uses fewer LoC than any of those other tutorials. It's far faster than other approaches. So why would anyone suggest using something else??

The only valid excuse I can muster is if you're not using Babel and your code needs to run on IE. But if you're not using Babel and your code needs to run on IE, you have a lot bigger problems than de-duping arrays.

So the next time you see someone hawking a "How To De-Dupe An Array In JavaScript" tutorial, and theydon't useSet(), please do us all a favor - and tell them to get the hell off the interwebs, before they hurt themselves or others.

Top comments(6)

Subscribe
pic
Create template

Templates let you quickly answer FAQs or store snippets for re-use.

Dismiss
CollapseExpand
 
monfernape profile image
Usman Khalil
Web Developer. Dead
  • Location
    Multan
  • Education
    Bachelor Mechanical Engineering
  • Work
    Frontend engineer at Copilot
  • Joined

Does that work with array of objects, having similar properties?

CollapseExpand
 
bytebodger profile image
Adam Nathaniel Davis
React acolyte, jack-of-all-(programming)trades, full-stack developer
  • Email
  • Location
    New Orleans, LA
  • Work
    Frontend Software Engineer
  • Joined
• Edited on• Edited

As Vesa demonstrated, this only works on scalar values. The reason is pretty simple:

console.log({a:1}==={a:1});// false
Enter fullscreen modeExit fullscreen mode

You could kinda sorta get around this using the oldJSON.stringify() hack:

constset1=newSet([{a:1},{a:1}]);console.log(set1.size);// 2constset2=newSet([JSON.stringify({a:1}),JSON.stringify({a:1})]);console.log(set2.size);// 1constset3=newSet([['a'],['a']]);console.log(set3.size);// 2constset4=newSet([JSON.stringify(['a']),JSON.stringify(['a'])]);console.log(set4.size);// 1
Enter fullscreen modeExit fullscreen mode

Although that only "works" if the keys/elements are in the same order. It would also fail in any scenario whereJSON.stringify() would fail - e.g., if the objects/arrays containfunctions.

CollapseExpand
 
kretaceous profile image
Abhijit Hota
I love almost everything about computers, specifically software. Diving deep into Web development and cloud and loving it! 💛👨‍💻
  • Location
    India
  • Education
    Indian Institute of Technology, Madras
  • Joined

Came here to comment that.@bytebodger you might want to add that as a catch.

Because this particular gotcha has been a pain for me earlier.

CollapseExpand
 
bytebodger profile image
Adam Nathaniel Davis
React acolyte, jack-of-all-(programming)trades, full-stack developer
  • Email
  • Location
    New Orleans, LA
  • Work
    Frontend Software Engineer
  • Joined
• Edited on• Edited

I did edit the original article slightly to indicate that it works withscalar (primitive) values. But I hadn't really thought of this as much of a "gotcha", because two objects thatlook the same... are not the same.

FWIW, this approach actually works with arrays and objects,if they are truly the same array/object. In other words, you can do something like this:

constanObject={one:'uno',two:'dos'};constanotherObject=anObject;constanArray=[anObject,anotherObject];constnoDupes=[...newSet(anArray)];
Enter fullscreen modeExit fullscreen mode

And the resultingnoDupes array will only contain one object. But if you do this:

constanObject={one:'uno',two:'dos'};constanotherObject={one:'uno',two:'dos'};constanArray=[anObject,anotherObject];constnoDupes=[...newSet(anArray)];
Enter fullscreen modeExit fullscreen mode

noDupes will contain two objects. Because, even thoughanObject andanotherObject look the same to our eye, they are not the samevalue.

CollapseExpand
 
kwstannard profile image
KW Stannard
Developer. Design pattern enthusiast. Tilting at Agile practices.
  • Work
    Lead Engineer at Andros.co
  • Joined

I do this with Ruby too.

Are you sure you want to hide this comment? It will become hidden in your post, but will still be visible via the comment'spermalink.

For further actions, you may consider blocking this person and/orreporting abuse

React acolyte, jack-of-all-(programming)trades, full-stack developer
  • Location
    New Orleans, LA
  • Work
    Frontend Software Engineer
  • Joined

More fromAdam Nathaniel Davis

DEV Community

We're a place where coders share, stay up-to-date and grow their careers.

Log in Create account

[8]ページ先頭

©2009-2025 Movatter.jp