remove duplicates delimited string batch

Question

Done a quick search, but can't get to the bottom of this one. I want to remove any duplicates from a string with values separated by commas. eg

set _str=1,2,3,1,5,4,2

Processing of this would populate a new variable with the 'un-duped' list. eg

echo %_finalstr%

Would produce:

1,2,3,5,4

Added complication _str can be any number of components (ie it could be 1 value, it could be 200). It could also be null. No problems with special chars though - it will all be in the format of number, comma, number, comma etc. The values could also be in any order.

Any ideas?

Answer 1

A way :

@echo off
set "$str=1,2,3,1,5,4,2"

for %%a in (%$str%) do set "##%%a=%%a"
setlocal enabledelayedexpansion
for /f "tokens=2 delims==" %%a in ('set ##') do set "$FinalStr=!$FinalStr!,%%a"
set "$FinalStr=!$FinalStr:~1!"
echo !$FinalStr!

Answer 2

@ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION
SET "string=1,2,3,5,4,2"
CALL :process
SET "string=1,2,3,5,4,2,1,1,1,1,1,1,1,1,107,221,3,2,13,2,13,2,13,2,13,2,13,2,13,2,13,2,13,2,13,2,13,2,13,2,13,2,13,2,13,2,13,2,13,2,13,2,13,2,13,2,13,2,13,2,13,2,13,2,13,2,13,2,13,2,13,2,13,2,13,2,1"
CALL :process
SET "string="
CALL :process
SET "string=3"
CALL :process
SET "string=3,3"
CALL :process
GOTO :eof

:process
SET "newstring=,"
IF DEFINED string FOR %%a IN (%string%) DO (SET "newstring=!newstring:,%%a,=,!%%a,")
SET "newstring=%newstring:~1,-1%"
ECHO oldstring:%string%:
ECHO newstring:%newstring%:
GOTO :EOF

Another way, with inbuilt demonstration

edit --- explanation.

Set newstring to a single comma. This ensures newstring both starts and ends with a comma.

The for iterates through each element in the string; the commas are seen as separators.

For each element n found, replaced any ,n, with , then add n, to the end.

Result: list of unique elements, surrounded by , - so remove the first and last characters and done.

Note that delayedexpansion and the use of ! facilitates this method, but

IF DEFINED string FOR %%a IN (%string%) DO CALL SET "newstring=%%newstring:,%%a,=,%%%%a,"

could be used regardless of delayedexpansion state.

The quotes are for terminal-space-suppression, following standard practice.

Answer 3

The Batch file below preserve the original order of the elements, as requested by the OP:

 set _str=1,2,3,1,5,4,2 

Processing of this would populate a new variable with the 'un-duped' list. eg

 echo %_finalstr% 

Would produce:

1,2,3,5,4

@echo off
setlocal EnableDelayedExpansion

set _str=1,2,3,1,5,4,2
REM set "_str="

set "_finalstr=,"
for %%a in (%_str%) do if "!_finalstr:,%%a,=!" equ "!_finalstr!" set "_finalstr=!_finalstr!%%a,"
set "_finalstr=%_finalstr:~1,-1%"
echo Oldstring: "%_str%"
echo Newstring: "%_finalstr%"