Skip to main content

Delete duplicate records from table even when there is no key field

Hi all,yesterday only i was discussing with my colleague Kamlesh Samnani who is MCTS about deleting records which does not have any primary key field based on which we can delete the data (actually he only fired this question yesterday to me)

so i thought to write a blog post about it,there are many posts which talk about deleting duplicate records from the table which has some key field,so i started working on the query and what i got is what i am gonna write here.

Lets start creating the table and after inserting some data in it,we will fire our query in that and will check the output as well

CREATE TABLE [dbo].[Employee](
[EmployeeId] [int] NULL,
[Salary] [money] NULL
) ON [PRIMARY]

INSERT INTO Employee VALUES(1,1000)
INSERT INTO Employee VALUES(1,2000)
INSERT INTO Employee VALUES(1,3000)
INSERT INTO Employee VALUES(2,2000)
INSERT INTO Employee VALUES(3,3000)
INSERT INTO Employee VALUES(3,4000)
INSERT INTO Employee VALUES(4,4000)
INSERT INTO Employee VALUES(4,5000)


after creating table and inserting some data table will look like following



Now we will try solving the problem using two method,first one is using simple query which is quite interesting (It was new for me when i created this query and performed) and second approach is using CTE (Common Table Expression)

1) Using Simple Query with Row_Number()

DELETE temp FROM
  (
      SELECT id FROM
      (
        SELECT row_number() over (PARTITION BY EmployeeId order by EmployeeId) AS   
        id, EmployeeId,Salary 
        FROM Employee
      ) AS t WHERE id > 1
   ) temp
   SELECT * FROM Employee

Here what row_number() will do is will create numbers for each of the row,now we have PARTITION BY EmployeeId so it will reset the number when EmployeeId changes,so on every new EmployeeId there will be number generated from 1,so we can directly delete data which are > 1,because they are duplicate,now in case if we do not have any duplicate record this row_number() will generated only once and it will never be reset,so in that case no records would be having same number generated

2) Using CTE

;WITH temp (uniquid,id,salary) AS 
 (
    SELECT row_number() OVER (PARTITION BY EmployeeId order by EmployeeId) AS id, 
    EmployeeId,Salary 
    FROM Employee
 )
 DELETE FROM temp WHERE uniquid>1
 SELECT * FROM Employee

Output




Both of the query will produce the desire output and will delete the duplicate records

I hope this is interesting one right,honestly i was not knowing until i performed it my self yesterday,so you can also hands on...

Comments

  1. I left IT befor some years.but this query reminds me of our CMC days.

    ReplyDelete

Post a Comment

Popular posts from this blog

One or more exceptions occurred while processing the subscribers to the 'item:creating' event

I was recently installing the packages from one of the QA environment to my local Sitecore instance, "Media library package" to be precise, And it started giving me this below error One or more exceptions occurred while processing the subscribers to the 'item:creating' event Looking at the sitecore logs, it gave me more info on the context and the actual inner exception was following Solution: 'Name' should consist only of letters, digits, dashes or underscore Now it was evident that some of my file names were violating the naming rule, I could see in the log just before exception from where the installer stopped creating items, and that file name had round braces "(" and ")" at the end of it, with my surprise I was able to create the item with those name in the content tree, but below was the solution for it, Solution I am using SC 9.0.1 and in that Go to Sitecore.Marketing.config file residing in "App_Config\Sitecore\Mar...

401.1 Unauthorized with windows authentication error code 0xc000006d

How many of you have faced this hosting issue when you do everything what it takes to run the site with windows authentication but still you are getting the same error again and again? If you think you also have faced the same issue and you tired of reading MSDN KBs for it and still have not found the issue (If KB has solved the issue, well and good, if not you can try this trick),Please Read below Typical scenario In typical hosting with IIS, i did every possible things like enabling windows authentication, changing it in web.config, configuring connection pool, authorization rules, it asks me for window authentication login and despite of entering correct credentials it always fails and keeps on asking for login, and when pressed cancel it gives 401.1 with 0xc000006d error code Solution (Which worked for me at-least after trying for almost 6-9 hrs) You need to change the Loop Back Check in registry so that it allows the host names which you are giving in url are allowed and au...

Error in Sitecore 10.3 XM with CDs - Could not find configuration node: database/database[@id='master']

Hi Team, Recently we came across two distinct scenarios with same error message.  1) We started getting below error when we try to hit our layout service API endpoint. If we hit end point of CM, it was working fine and when we try to hit end point of CD it was giving this error, It was clear there is something in backend expecting master DB instead of WEB DB In first scenario my colleague  Jatin  also got same issue while setting up docker instance for 10.3, and my friend  Akshay Barve  and I had the same observation. 2) Second scenario I was having the same exception, It was another project on 10.3 XM with CD, It was one of the migration project so we were migrating from 10.2 to 10.3, Older site was not giving any error but new site the moment we hit, it was giving same exception in the log Issue in second scenario was in site grouping there was only site and both CM and CD URLs were given in host name and had WEB db as database, so basically there was no previ...