java8 lamda表达式实现集合对象重复数据过滤的方式

实现重复数据过滤的实现方式

• 1.实现对于简单数组的重复数据的过滤

  JSONArray arr = JSONArray.parseArray("[ \"13111111111111\", \"13111111111111\" ]"); # 第一种 return arr.stream().distinct().collect(Collectors.toList()); # 第二种 return arr.stream().collect(Collectors.toSet());

思路: 1. 根据distinct()函数进行过滤

• 2.实现复杂嵌套对象的重复数据的过滤

  JSONArray array = JSONArray.parseArray("[{\"value\":{\"value\":\"wang\"}},{\"value\":{\"value\":\"zhang\"}},{\"value\":{\"value\":\"wang\"}}]"); array.stream().filter(d->((JSONObject)d).get("value")!=null &&((JSONObject)((JSONObject)d).get("value")).get("value")!=null).collect(Collectors.groupingBy(m->((JSONObject)((JSONObject)m).get("value")))).entrySet().stream().map(x->{ List list = x.getValue(); if(list.size()>1){ x.setValue(x.getValue().subList(0,1)); } return x.getValue(); }).flatMap(m->m.stream()).collect(Collectors.toList())

思路: 1.过滤下非空数据 2.获取嵌套对象目标数据值，并根据这个值进行分组 3.对分组结果中长度大于1的，截取第一个 4.扁平化处理

实现排序和日期合并分组的实现方式

1. 过滤数组下对象中特定字段名为某某的结果（保留)

((JSONArrray)data).stream().filter(m->((JSONObject)m).get("column")!=null && "col_1".equals(((JSONObject)m).get("column"))).collector(Collectors.toList()

样例:

[ {"column":"col_1", "result":"test"}, {"column":"col_2", "result"：test} ]

输出:

[ {"column":"col_1","result":"test"} ]

2. 实现按对象下数组中对象中的键进行降序排序，并输出对象

((JSONObject)data).entrySet().stream() .peek(m->((JSONArray)m.getValue())) .sorted(Comparator.comparing(x->((JSONObject)x).get("create_time")).reversed()).collect(Collectors.toMap(b->b.getKey(),d->d.getKey()));

样例:

{ "测试1":[ {"create_time":"2020-05-20","data":{"type":1}}, {"create_time":"2020-06-15":"data":{"type":1}} ], "测试2":[{ "create_time":"2020-06-12","data":{} }], }

输出:

{ "测试1":[ {"create_time":"2020-06-15","data":{"type":1}}, {"create_time":"2020-05-20":"data":{"type":1}} ], "测试2":[{ "create_time":"2020-06-12","data":{} }], }

3. 实现根据数组中对象中时间中日期进行分组，

((JSONArray)data).stream().collect(Collectors.groupingBy(w->LocalDateTime.parse(((JSONObject))w).get("create_time").toString(),DateTimeFormatter.ofPattern(DateUtil.DATE_FORMAT_01)).toLocalDate()).ToString()))));

样例:

[ {"create_time":"2020-05-24 12:20:00","result":{}}, {"create_time":"2020-05-24 11:30:13","result":{}}, {"create_time":"2020-05-30 09:08:01","result":{}}, {"create_time":"2020-06-15 11:30:13","result":{}}, {"create_time":"2020-06-15 19:30:13","result":{}}, ]

输出:

{ "2020-05-24": [ {"create_time":"2020-05-24 12:20:00","result":{}}, {"create_time":"2020-05-24 11:30:13","result":{}} ], "2020-05-30": [ {"create_time":"2020-05-30 09:08:01","result":{}} ], "2020=06-15": [ {"create_time":"2020-06-15 19:30:13","result":{}} ] }

4. 将不同对象合并为一个数组

((JSONObject)data).entrySet().stream().flatMap(n->((JSONArray)n.getValue()).stream()).limit(50).collect(Collectors.toList())

样例:

{ "测试1":[ {"create_time":"2020-05-20 12:20:00"}, {"create_time":"2020-06-12 15:00:00"} ], "测试2": [ {"create_time":"2020-05-16 11:30:00"} ] }

输出:

[ {"create_time":"2020-05-20 12:20:00"}, {"create_time":"2020-06-12 15:00:00"}, {"create_time":"2020-05-16 11:30:00"} ]