Version: 3.2.2 Update

Version: 3.2.1 Update
2024-06-19 22:20:46 +08:00 · 2024-06-19 22:16:57 +08:00 · 2024-05-30 16:01:25 +08:00 · 2024-05-30 15:56:49 +08:00 · 2024-05-30 14:42:26 +08:00 · 2024-05-30 14:37:01 +08:00
23 changed files with 764 additions and 317 deletions
--- a/README.md
+++ b/README.md
@@ -1,7 +1,7 @@
 <div align="center">
 <img src="images/logo.png" style="width: 20%" />
 <h4><a href="https://gh0st.cn/HaE/">赋能白帽，高效作战！</a></h4>
-<h5>第一作者： <a href="https://github.com/gh0stkey">EvilChen</a>（中孚信息元亨实验室）, 第二作者： <a href="https://github.com/0chencc">0chencc</a>（米斯特安全团队）</h5>
+<h5>第一作者： <a href="https://github.com/gh0stkey">EvilChen</a>（中孚信息元亨实验室）<br>第二作者： <a href="https://github.com/0chencc">0chencc</a>（米斯特安全团队）<br>第三作者： <a href="https://github.com/vaycore">vaycore</a>（独立安全研究员）</h5>
 </div>

 ## 项目介绍
@@ -14,7 +14,7 @@

 1. 由于HaE 3.0版本开始采用`Montoya API`进行开发，因此使用新版HaE需要升级你的BurpSuite版本（>=2023.12.1）。
 2. 由于HaE 2.6版本后对规则字段进行了更新，因此无法适配<=2.6版本的规则，请用户自行前往[规则转换页面](https://gh0st.cn/HaE/ConversionRule.html)进行转换。
-3. HaE官方规则库存放在[Github](https://raw.githubusercontent.com/gh0stkey/HaE/gh-pages/Rules.yml)上，因此默认加载HaE官方规则库需使用代理（BApp审核不允许使用CDN）。
+3. HaE官方规则库存放在[Github](https://raw.githubusercontent.com/gh0stkey/HaE/gh-pages/Rules.yml)上，因此点击`Update`升级HaE官方规则库时需使用代理（BApp审核考虑安全性，不允许使用CDN）。
 4. 自定义HaE规则必须用左右括号`()`将所需提取的表达式内容包含，例如你要匹配一个**Shiro应用**的响应报文，正常匹配规则为`rememberMe=delete`，在HaE的规则中就需要变成`(rememberMe=delete)`。

 ## 使用方法
@@ -46,14 +46,12 @@ HaE目前的规则一共有8个字段，详细的含义如下所示：

 ## 优势特点

-1. 精细配置：高度自由的配置选项，以满足各类精细化场景需求。
-2. 分类标签：使用标签对规则进行分类，便于管理和组织规则。
-3. 高亮标记：在HTTP History页面，通过颜色高亮和注释判断请求的价值。
-4. 易读配置：使用易读的YAML格式存储配置文件，方便阅读和修改。
-5. 数据集合：将匹配到的数据、请求和响应集中在数据面板中，提高测试和梳理效率。
-6. 简洁可视：清晰可视的界面设计，更轻松地了解和配置HaE，操作简单、使用便捷。
-7. 颜色升级：内置颜色升级算法，避免“屠龙者终成恶龙”场景，突出最具价值的请求。
-8. 实战规则：官方规则库是基于实战化场景总结输出，提升数据发现的有效性、精准性。
+1. **功能**：通过对HTTP报文的颜色高亮、注释和提取，帮助使用者获取有意义的信息，**聚焦高价值报文**。
+2. **界面**：清晰可视的界面设计，以及**简洁的界面交互**，帮助使用者更轻松的了解和配置项目，**避免`多按钮`式的复杂体验**。
+3. **查询**：将HTTP报文的高亮、注释和提取到的相关信息**集中在一个数据面板**，可以一键查询、提取信息，从而提高测试和梳理效率。
+4. **算法**：内置高亮颜色的升级算法，当出现相同颜色时**会自动向上升级一个颜色**进行标记，**避免`屠龙者终成恶龙`场景**。
+5. **管理**：支持对数据的一键导出、导入，以**自定义`.hae`文件的方式**进行项目数据存储，**便于存储和共享项目数据**。
+6. **实战**：官方规则库和规则字段作用功能，都是**基于实战化场景总结输出**的，**以此提高数据的有效性、精准性发现**。

 | 界面名称                  | 界面展示                                              |
 | ------------------------ | ---------------------------------------------------- |
@@ -62,9 +60,7 @@ HaE目前的规则一共有8个字段，详细的含义如下所示：
 | Databoard（数据集合） | <img src="images/databoard.png" style="width: 80%" /> |
 | MarkInfo（数据展示） | <img src="images/markinfo.png" style="width: 80%" /> |

-## 文末随笔
-
-正义感是一个不可丢失的东西。
+## 支持项目

 如果你觉得HaE好用，可以打赏一下作者，给作者持续更新下去的动力！

--- a/build.gradle
+++ b/build.gradle
@@ -20,8 +20,8 @@ sourceSets {
 dependencies {
    implementation 'net.portswigger.burp.extensions:montoya-api:2023.12.1'
    implementation 'org.yaml:snakeyaml:2.0'
-    implementation 'net.sourceforge.jregex:jregex:1.2_01'
    implementation 'dk.brics.automaton:automaton:1.11-8'
+    implementation 'com.github.ben-manes.caffeine:caffeine:3.1.8'
 }

 test {
--- a/images/config.png
+++ b/images/config.png
--- a/images/databoard.png
+++ b/images/databoard.png
--- a/images/rules.png
+++ b/images/rules.png
--- a/src/main/java/hae/Config.java
+++ b/src/main/java/hae/Config.java
@@ -54,4 +54,6 @@ public class Config {
    public static Map<String, Object[][]> globalRules = new HashMap<>();

    public static ConcurrentHashMap<String, Map<String, List<String>>> globalDataMap = new ConcurrentHashMap<>();
+
+    public static ConcurrentHashMap<String, Map<String, Object>> globalHostHashMap = new ConcurrentHashMap<>();
 }
--- a/src/main/java/hae/HaE.java
+++ b/src/main/java/hae/HaE.java
@@ -2,7 +2,9 @@ package hae;

 import burp.api.montoya.BurpExtension;
 import burp.api.montoya.MontoyaApi;
+import burp.api.montoya.extension.ExtensionUnloadingHandler;
 import burp.api.montoya.logging.Logging;
+import hae.cache.CachePool;
 import hae.component.Main;
 import hae.component.board.message.MessageTableModel;
 import hae.instances.editor.RequestEditor;
@@ -16,13 +18,13 @@ public class HaE implements BurpExtension {
    @Override
    public void initialize(MontoyaApi api) {
        // 设置扩展名称
-        String version = "3.2";
+        String version = "3.2.2";
        api.extension().setName(String.format("HaE (%s) - Highlighter and Extractor", version));

        // 加载扩展后输出的项目信息
        Logging logging = api.logging();
        logging.logToOutput("[ HACK THE WORLD - TO DO IT ]");
-        logging.logToOutput("[#] Author: EvilChen && 0chencc");
+        logging.logToOutput("[#] Author: EvilChen && 0chencc && vaycore");
        logging.logToOutput("[#] Github: https://github.com/gh0stkey/HaE");

        // 配置文件加载
@@ -43,5 +45,14 @@ public class HaE implements BurpExtension {
        api.userInterface().registerHttpRequestEditorProvider(new RequestEditor(api, configLoader));
        api.userInterface().registerHttpResponseEditorProvider(new ResponseEditor(api, configLoader));
        api.userInterface().registerWebSocketMessageEditorProvider(new WebSocketEditor(api));
+
+        api.extension().registerUnloadingHandler(new ExtensionUnloadingHandler() {
+            @Override
+            public void extensionUnloaded() {
+                // 卸载清空数据
+                Config.globalDataMap.clear();
+                CachePool.clear();
+            }
+        });
    }
 }
--- a/src/main/java/hae/component/board/Databoard.java
+++ b/src/main/java/hae/component/board/Databoard.java
@@ -1,10 +1,6 @@
 package hae.component.board;

 import burp.api.montoya.MontoyaApi;
-import burp.api.montoya.http.HttpService;
-import burp.api.montoya.http.message.HttpRequestResponse;
-import burp.api.montoya.http.message.requests.HttpRequest;
-import burp.api.montoya.http.message.responses.HttpResponse;
 import hae.Config;
 import hae.component.board.message.MessageEntry;
 import hae.component.board.message.MessageTableModel;
@@ -27,7 +23,11 @@ import java.awt.event.*;
 import java.io.File;
 import java.util.List;
 import java.util.*;
+import java.util.concurrent.Callable;
 import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.function.Function;
 import java.util.stream.Collectors;

 public class Databoard extends JPanel {
@@ -35,15 +35,22 @@ public class Databoard extends JPanel {
    private final ConfigLoader configLoader;
    private final ProjectProcessor projectProcessor;
    private final MessageTableModel messageTableModel;
+
    private JTextField hostTextField;
    private JTabbedPane dataTabbedPane;
    private JSplitPane splitPane;
    private MessageTable messageTable;
+    private JProgressBar progressBar;

    private static Boolean isMatchHost = false;
    private final DefaultComboBoxModel comboBoxModel = new DefaultComboBoxModel();
    private final JComboBox hostComboBox = new JComboBox(comboBoxModel);

+    private SwingWorker<Boolean, Void> handleComboBoxWorker;
+    private SwingWorker<Void, Void> applyHostFilterWorker;
+    private SwingWorker<List<String>, Void> exportActionWorker;
+    private SwingWorker<List<String>, Void> importActionWorker;
+
    public Databoard(MontoyaApi api, ConfigLoader configLoader, MessageTableModel messageTableModel) {
        this.api = api;
        this.configLoader = configLoader;
@@ -56,9 +63,9 @@ public class Databoard extends JPanel {
    private void initComponents() {
        setLayout(new GridBagLayout());
        ((GridBagLayout) getLayout()).columnWidths = new int[]{25, 0, 0, 0, 20, 0};
-        ((GridBagLayout) getLayout()).rowHeights = new int[]{0, 65, 20, 0};
+        ((GridBagLayout) getLayout()).rowHeights = new int[]{0, 65, 20, 25, 0};
        ((GridBagLayout) getLayout()).columnWeights = new double[]{0.0, 0.0, 1.0, 0.0, 0.0, 1.0E-4};
-        ((GridBagLayout) getLayout()).rowWeights = new double[]{0.0, 1.0, 0.0, 1.0E-4};
+        ((GridBagLayout) getLayout()).rowWeights = new double[]{0.0, 1.0, 0.0, 0.0, 1.0E-4};

        JLabel hostLabel = new JLabel("Host:");

@@ -76,7 +83,10 @@ public class Databoard extends JPanel {

        hostTextField = new JTextField();
        splitPane = new JSplitPane(JSplitPane.HORIZONTAL_SPLIT);
+
        dataTabbedPane = new JTabbedPane(JTabbedPane.TOP);
+        dataTabbedPane.setPreferredSize(new Dimension(500, 0));
+        dataTabbedPane.setTabLayoutPolicy(JTabbedPane.SCROLL_TAB_LAYOUT);

        actionButton.addActionListener(e -> {
            int x = 0;
@@ -88,6 +98,8 @@ public class Databoard extends JPanel {
        exportButton.addActionListener(this::exportActionPerformed);
        importButton.addActionListener(this::importActionPerformed);

+        progressBar = new JProgressBar();
+
        splitPane.addComponentListener(new ComponentAdapter() {
            @Override
            public void componentResized(ComponentEvent e) {
@@ -96,6 +108,7 @@ public class Databoard extends JPanel {
        });

        splitPane.setVisible(false);
+        progressBar.setVisible(false);

        add(hostLabel, new GridBagConstraints(1, 0, 1, 1, 0.0, 0.0, GridBagConstraints.CENTER, GridBagConstraints.BOTH,
                new Insets(8, 0, 5, 5), 0, 0));
@@ -103,9 +116,13 @@ public class Databoard extends JPanel {
                new Insets(8, 0, 5, 5), 0, 0));
        add(actionButton, new GridBagConstraints(3, 0, 1, 1, 0.0, 0.0, GridBagConstraints.CENTER, GridBagConstraints.BOTH,
                new Insets(8, 0, 5, 5), 0, 0));
-        add(splitPane, new GridBagConstraints(1, 1, 3, 3, 0.0, 0.0,
+
+        add(splitPane, new GridBagConstraints(1, 1, 3, 1, 0.0, 1.0,
                GridBagConstraints.CENTER, GridBagConstraints.BOTH,
-                new Insets(8, 0, 5, 5), 0, 0));
+                new Insets(0, 5, 0, 5), 0, 0));
+        add(progressBar, new GridBagConstraints(1, 3, 3, 1, 1.0, 0.0,
+                GridBagConstraints.CENTER, GridBagConstraints.HORIZONTAL,
+                new Insets(0, 5, 0, 5), 0, 0));
        hostComboBox.setMaximumRowCount(5);
        add(hostComboBox, new GridBagConstraints(2, 0, 1, 1, 0.0, 0.0, GridBagConstraints.CENTER, GridBagConstraints.BOTH,
                new Insets(8, 0, 5, 5), 0, 0));
@@ -125,6 +142,19 @@ public class Databoard extends JPanel {
        columnModel.getColumn(5).setPreferredWidth((int) (totalWidth * 0.1));
    }

+    private void setProgressBar(boolean status) {
+        progressBar.setIndeterminate(status);
+        if (!status) {
+            progressBar.setMaximum(100);
+            progressBar.setString("OK");
+            progressBar.setStringPainted(true);
+            progressBar.setValue(progressBar.getMaximum());
+        } else {
+            progressBar.setString("Loading...");
+            progressBar.setStringPainted(true);
+        }
+    }
+
    private void setAutoMatch() {
        hostComboBox.setSelectedItem(null);
        hostComboBox.addActionListener(this::handleComboBoxAction);
@@ -157,9 +187,49 @@ public class Databoard extends JPanel {

    private void handleComboBoxAction(ActionEvent e) {
        if (!isMatchHost && hostComboBox.getSelectedItem() != null) {
+            progressBar.setVisible(true);
+            setProgressBar(true);
            String selectedHost = hostComboBox.getSelectedItem().toString();
+
+            if (getHostByList().contains(selectedHost)) {
                hostTextField.setText(selectedHost);
-            populateTabbedPaneByHost(selectedHost);
+
+                if (handleComboBoxWorker != null && !handleComboBoxWorker.isDone()) {
+                    handleComboBoxWorker.cancel(true);
+                }
+
+                handleComboBoxWorker = new SwingWorker<Boolean, Void>() {
+                    @Override
+                    protected Boolean doInBackground() {
+                        return populateTabbedPaneByHost(selectedHost);
+                    }
+
+                    @Override
+                    protected void done() {
+                        if (!isCancelled()) {
+                            try {
+                                boolean status = get();
+                                if (status) {
+                                    JSplitPane messageSplitPane = messageTableModel.getSplitPane();
+                                    splitPane.setLeftComponent(dataTabbedPane);
+                                    splitPane.setRightComponent(messageSplitPane);
+                                    messageTable = messageTableModel.getMessageTable();
+                                    resizePanel();
+
+                                    splitPane.setVisible(true);
+                                    hostTextField.setText(selectedHost);
+
+                                    hostComboBox.setPopupVisible(false);
+                                    applyHostFilter(selectedHost);
+                                }
+                            } catch (Exception ignored) {
+                            }
+                        }
+                    }
+                };
+
+                handleComboBoxWorker.execute();
+            }
        }
    }

@@ -178,7 +248,6 @@ public class Databoard extends JPanel {
        if (keyCode == KeyEvent.VK_ENTER) {
            isMatchHost = false;
            handleComboBoxAction(null);
-            hostComboBox.setPopupVisible(false);
        }

        if (keyCode == KeyEvent.VK_ESCAPE) {
@@ -188,10 +257,53 @@ public class Databoard extends JPanel {
        isMatchHost = false;
    }

+    private boolean populateTabbedPaneByHost(String selectedHost) {
+        ConcurrentHashMap<String, Map<String, List<String>>> dataMap = Config.globalDataMap;
+        Map<String, List<String>> selectedDataMap;
+
+        if (selectedHost.contains("*")) {
+            selectedDataMap = new HashMap<>();
+            dataMap.keySet().forEach(key -> {
+                if ((StringProcessor.matchesHostPattern(key, selectedHost) || selectedHost.equals("*")) && !key.contains("*")) {
+                    Map<String, List<String>> ruleMap = dataMap.get(key);
+                    for (String ruleKey : ruleMap.keySet()) {
+                        List<String> dataList = ruleMap.get(ruleKey);
+                        if (selectedDataMap.containsKey(ruleKey)) {
+                            List<String> mergedList = new ArrayList<>(selectedDataMap.get(ruleKey));
+                            mergedList.addAll(dataList);
+                            HashSet<String> uniqueSet = new HashSet<>(mergedList);
+                            selectedDataMap.put(ruleKey, new ArrayList<>(uniqueSet));
+                        } else {
+                            selectedDataMap.put(ruleKey, dataList);
+                        }
+                    }
+                }
+            });
+        } else {
+            selectedDataMap = dataMap.get(selectedHost);
+        }
+
+        if (!selectedDataMap.isEmpty()) {
+            dataTabbedPane.removeAll();
+
+            for (Map.Entry<String, List<String>> entry : selectedDataMap.entrySet()) {
+                String tabTitle = String.format("%s (%s)", entry.getKey(), entry.getValue().size());
+                Datatable datatablePanel = new Datatable(api, entry.getKey(), entry.getValue());
+                datatablePanel.setTableListener(messageTableModel);
+                dataTabbedPane.addTab(tabTitle, datatablePanel);
+            }
+
+            return true;
+        }
+
+        return false;
+    }
+
    private void filterComboBoxList() {
        isMatchHost = true;
        comboBoxModel.removeAllElements();
        String input = hostTextField.getText().toLowerCase();
+
        if (!input.isEmpty()) {
            for (String host : getHostByList()) {
                String lowerCaseHost = host.toLowerCase();
@@ -210,83 +322,45 @@ public class Databoard extends JPanel {
        isMatchHost = false;
    }

-    private void populateTabbedPaneByHost(String selectedHost) {
-        if (!Objects.equals(selectedHost, "")) {
-            ConcurrentHashMap<String, Map<String, List<String>>> dataMap = Config.globalDataMap;
-            Map<String, List<String>> selectedDataMap;
-
-            dataTabbedPane.removeAll();
-            dataTabbedPane.setPreferredSize(new Dimension(500, 0));
-            dataTabbedPane.setTabLayoutPolicy(JTabbedPane.SCROLL_TAB_LAYOUT);
-            splitPane.setLeftComponent(dataTabbedPane);
-
-            if (selectedHost.contains("*")) {
-                // 通配符数据
-                selectedDataMap = new HashMap<>();
-                for (String key : dataMap.keySet()) {
-                    if ((StringProcessor.matchesHostPattern(key, selectedHost) || selectedHost.equals("*")) && !key.contains("*")) {
-                        Map<String, List<String>> ruleMap = dataMap.get(key);
-                        for (String ruleKey : ruleMap.keySet()) {
-                            List<String> dataList = ruleMap.get(ruleKey);
-                            if (selectedDataMap.containsKey(ruleKey)) {
-                                List<String> mergedList = new ArrayList<>(selectedDataMap.get(ruleKey));
-                                mergedList.addAll(dataList);
-                                HashSet<String> uniqueSet = new HashSet<>(mergedList);
-                                selectedDataMap.put(ruleKey, new ArrayList<>(uniqueSet));
-                            } else {
-                                selectedDataMap.put(ruleKey, dataList);
-                            }
-                        }
-                    }
-                }
-            } else {
-                selectedDataMap = dataMap.get(selectedHost);
-            }
-
-            for (Map.Entry<String, List<String>> entry : selectedDataMap.entrySet()) {
-                String tabTitle = String.format("%s (%s)", entry.getKey(), entry.getValue().size());
-                Datatable datatablePanel = new Datatable(api, entry.getKey(), entry.getValue());
-                datatablePanel.setTableListener(messageTableModel);
-                dataTabbedPane.addTab(tabTitle, datatablePanel);
-            }
-
-            // 展示请求消息表单
-            JSplitPane messageSplitPane = messageTableModel.getSplitPane();
-            this.splitPane.setRightComponent(messageSplitPane);
-            messageTable = messageTableModel.getMessageTable();
-
-            resizePanel();
-            splitPane.setVisible(true);
-
-            applyHostFilter(selectedHost);
-            hostTextField.setText(selectedHost);
-        }
-    }
-
    private void applyHostFilter(String filterText) {
        TableRowSorter<TableModel> sorter = (TableRowSorter<TableModel>) messageTable.getRowSorter();
-
        String cleanedText = StringProcessor.replaceFirstOccurrence(filterText, "*.", "");

+        if (applyHostFilterWorker != null && !applyHostFilterWorker.isDone()) {
+            applyHostFilterWorker.cancel(true);
+        }
+
+        applyHostFilterWorker = new SwingWorker<Void, Void>() {
+            @Override
+            protected Void doInBackground() throws Exception {
                RowFilter<Object, Object> rowFilter = new RowFilter<Object, Object>() {
                    public boolean include(Entry<?, ?> entry) {
                        if (cleanedText.equals("*")) {
                            return true;
                        } else {
                            String host = StringProcessor.getHostByUrl((String) entry.getValue(1));
-
                            return StringProcessor.matchesHostPattern(host, filterText);
                        }
                    }
                };

                sorter.setRowFilter(rowFilter);
-
                messageTableModel.applyHostFilter(filterText);
+
+                return null;
+            }
+
+            @Override
+            protected void done() {
+                setProgressBar(false);
+            }
+        };
+
+        applyHostFilterWorker.execute();
    }

    private List<String> getHostByList() {
-        if (!(Config.globalDataMap.keySet().size() == 1 && Config.globalDataMap.keySet().stream().anyMatch(key -> key.contains("*")))) {
+        if (!Config.globalDataMap.keySet().isEmpty()) {
            return new ArrayList<>(Config.globalDataMap.keySet());
        }
        return new ArrayList<>();
@@ -305,13 +379,31 @@ public class Databoard extends JPanel {
            return;
        }

-        ConcurrentHashMap<String, Map<String, List<String>>> dataMap = Config.globalDataMap;
-        List<String> taskStatusList = exportData(selectedHost, exportDir, dataMap);
+        if (exportActionWorker != null && !exportActionWorker.isDone()) {
+            exportActionWorker.cancel(true);
+        }

+        exportActionWorker = new SwingWorker<List<String>, Void>() {
+            @Override
+            protected List<String> doInBackground() {
+                ConcurrentHashMap<String, Map<String, List<String>>> dataMap = Config.globalDataMap;
+                return exportData(selectedHost, exportDir, dataMap);
+            }
+
+            @Override
+            protected void done() {
+                try {
+                    List<String> taskStatusList = get();
                    if (!taskStatusList.isEmpty()) {
                        String exportStatusMessage = String.format("Exported File List Status:\n%s", String.join("\n", taskStatusList));
-            JOptionPane.showConfirmDialog(null, exportStatusMessage, "Info", JOptionPane.YES_OPTION);
+                        JOptionPane.showConfirmDialog(Databoard.this, exportStatusMessage, "Info", JOptionPane.YES_OPTION);
                    }
+                } catch (Exception ignored) {
+                }
+            }
+        };
+
+        exportActionWorker.execute();
    }

    private List<String> exportData(String selectedHost, String exportDir, Map<String, Map<String, List<String>>> dataMap) {
@@ -332,28 +424,63 @@ public class Databoard extends JPanel {
        }

        List<MessageEntry> messageEntryList = messageTableModel.getLogs();
-        Map<String, Map<String, String>> httpMap = messageEntryList.stream()
-                .filter(messageEntry -> !StringProcessor.getHostByUrl(messageEntry.getUrl()).isEmpty())
-                .filter(messageEntry -> StringProcessor.getHostByUrl(messageEntry.getUrl()).equals(key))
+
+        Map<MessageEntry, String> entryUUIDMap = messageEntryList.stream()
                .collect(Collectors.toMap(
-                        MessageEntry::getUrl,
-                        this::createHttpItemMap,
-                        (existing, replacement) -> existing
+                        messageEntry -> messageEntry,
+                        messageEntry -> StringProcessor.getRandomUUID(),
+                        (existing, replacement) -> existing // 在冲突时保留现有的映射
                ));

+        Map<String, Map<String, Object>> httpMap = processEntries(
+                messageEntryList,
+                key,
+                entryUUIDMap,
+                this::createHttpItemMap
+        );
+
+        Map<String, Map<String, Object>> urlMap = processEntries(
+                messageEntryList,
+                key,
+                entryUUIDMap,
+                this::creteUrlItemMap
+        );
+
        String hostName = key.replace(":", "_");
-        String filename = String.format("%s/%s.hae", exportDir, hostName);
-        boolean createdStatus = projectProcessor.createHaeFile(filename, key, ruleMap, httpMap);
+        String filename = String.format("%s/%s-%s.hae", exportDir, StringProcessor.getCurrentTime(), hostName);
+        boolean createdStatus = projectProcessor.createHaeFile(filename, key, ruleMap, urlMap, httpMap);

        return String.format("Filename: %s, Status: %s", filename, createdStatus);
    }

-    private Map<String, String> createHttpItemMap(MessageEntry entry) {
-        Map<String, String> httpItemMap = new HashMap<>();
-        httpItemMap.put("comment", entry.getComment());
-        httpItemMap.put("color", entry.getColor());
-        httpItemMap.put("request", entry.getRequestResponse().request().toString());
-        httpItemMap.put("response", entry.getRequestResponse().response().toString());
+
+    private Map<String, Map<String, Object>> processEntries(List<MessageEntry> messageEntryList, String key, Map<MessageEntry, String> entryUUIDMap, Function<MessageEntry, Map<String, Object>> mapFunction) {
+        return messageEntryList.stream()
+                .filter(messageEntry -> !StringProcessor.getHostByUrl(messageEntry.getUrl()).isEmpty())
+                .filter(messageEntry -> StringProcessor.getHostByUrl(messageEntry.getUrl()).equals(key))
+                .collect(Collectors.toMap(
+                        entryUUIDMap::get,
+                        mapFunction,
+                        (existing, replacement) -> existing
+                ));
+    }
+
+    private Map<String, Object> creteUrlItemMap(MessageEntry entry) {
+        Map<String, Object> urlItemMap = new LinkedHashMap<>();
+        urlItemMap.put("url", entry.getUrl());
+        urlItemMap.put("method", entry.getMethod());
+        urlItemMap.put("status", entry.getStatus());
+        urlItemMap.put("length", entry.getLength());
+        urlItemMap.put("comment", entry.getComment());
+        urlItemMap.put("color", entry.getColor());
+        urlItemMap.put("size", String.valueOf(entry.getRequestResponse().request().toByteArray().length()));
+        return urlItemMap;
+    }
+
+    private Map<String, Object> createHttpItemMap(MessageEntry entry) {
+        Map<String, Object> httpItemMap = new LinkedHashMap<>();
+        httpItemMap.put("request", entry.getRequestResponse().request().toByteArray().getBytes());
+        httpItemMap.put("response", entry.getRequestResponse().response().toByteArray().getBytes());
        return httpItemMap;
    }

@@ -363,43 +490,74 @@ public class Databoard extends JPanel {
            return;
        }

-        List<String> filesWithExtension = findFilesWithExtension(new File(exportDir), ".hae");
-        List<String> taskStatusList = filesWithExtension.stream()
-                .map(this::importData)
-                .collect(Collectors.toList());
+        if (importActionWorker != null && !importActionWorker.isDone()) {
+            importActionWorker.cancel(true);
+        }

+        importActionWorker = new SwingWorker<List<String>, Void>() {
+            @Override
+            protected List<String> doInBackground() {
+                List<String> filesWithExtension = findFilesWithExtension(new File(exportDir), ".hae");
+                return filesWithExtension.stream()
+                        .map(Databoard.this::importData)
+                        .collect(Collectors.toList());
+            }
+
+            @Override
+            protected void done() {
+                try {
+                    List<String> taskStatusList = get();
                    if (!taskStatusList.isEmpty()) {
                        String importStatusMessage = "Imported File List Status:\n" + String.join("\n", taskStatusList);
-            JOptionPane.showConfirmDialog(null, importStatusMessage, "Info", JOptionPane.YES_OPTION);
+                        JOptionPane.showConfirmDialog(Databoard.this, importStatusMessage, "Info", JOptionPane.YES_OPTION);
                    }
+                } catch (Exception ignored) {
+                }
+            }
+        };
+
+        importActionWorker.execute();
    }

    private String importData(String filename) {
+        ExecutorService executor = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors() * 2);
+
        HaeFileContent haeFileContent = projectProcessor.readHaeFile(filename);
        boolean readStatus = haeFileContent != null;

+        List<Callable<Void>> tasks = new ArrayList<>();
+
        if (readStatus) {
+            try {
                String host = haeFileContent.getHost();
                haeFileContent.getDataMap().forEach((key, value) -> RegularMatcher.putDataToGlobalMap(host, key, value));

-            haeFileContent.getHttpMap().forEach((key, httpItemMap) -> {
-                String comment = httpItemMap.get("comment");
-                String color = httpItemMap.get("color");
-                HttpRequestResponse httpRequestResponse = createHttpRequestResponse(key, httpItemMap);
-                messageTableModel.add(httpRequestResponse, comment, color);
+                haeFileContent.getUrlMap().forEach((key, urlItemMap) -> {
+                    tasks.add(() -> {
+                        String url = urlItemMap.get("url");
+                        String comment = urlItemMap.get("comment");
+                        String color = urlItemMap.get("color");
+                        String length = urlItemMap.get("length");
+                        String method = urlItemMap.get("method");
+                        String status = urlItemMap.get("status");
+                        String path = haeFileContent.getHttpPath();
+
+                        messageTableModel.add(null, url, method, status, length, comment, color, key, path);
+                        return null;
                    });
+                });
+
+                executor.invokeAll(tasks);
+            } catch (Exception e) {
+                api.logging().logToError("importData: " + e.getMessage());
+            } finally {
+                executor.shutdown();
+            }
        }

        return String.format("Filename: %s, Status: %s", filename, readStatus);
    }

-    private HttpRequestResponse createHttpRequestResponse(String key, Map<String, String> httpItemMap) {
-        HttpService httpService = HttpService.httpService(key);
-        HttpRequest httpRequest = HttpRequest.httpRequest(httpService, httpItemMap.get("request"));
-        HttpResponse httpResponse = HttpResponse.httpResponse(httpItemMap.get("response"));
-        return HttpRequestResponse.httpRequestResponse(httpRequest, httpResponse);
-    }
-
    private List<String> findFilesWithExtension(File directory, String extension) {
        List<String> filePaths = new ArrayList<>();
        if (directory.isDirectory()) {
@@ -413,8 +571,9 @@ public class Databoard extends JPanel {
                    }
                }
            }
-        }
+        } else {
            filePaths.add(directory.getAbsolutePath());
+        }
        return filePaths;
    }

@@ -438,19 +597,41 @@ public class Databoard extends JPanel {
    }

    private void clearActionPerformed(ActionEvent e) {
-        int retCode = JOptionPane.showConfirmDialog(null, "Do you want to clear data?", "Info",
+        int retCode = JOptionPane.showConfirmDialog(this, "Do you want to clear data?", "Info",
                JOptionPane.YES_NO_OPTION);
        String host = hostTextField.getText();
        if (retCode == JOptionPane.YES_OPTION && !host.isEmpty()) {
            dataTabbedPane.removeAll();
            splitPane.setVisible(false);
+            progressBar.setVisible(false);

-            String cleanedHost = StringProcessor.replaceFirstOccurrence(host, "*.", "");
+            Config.globalDataMap.keySet().parallelStream().forEach(key -> {
+                if (StringProcessor.matchesHostPattern(key, host) || host.equals("*")) {
+                    Config.globalDataMap.remove(key);
+                }
+            });

-            if (host.contains("*")) {
-                Config.globalDataMap.keySet().removeIf(i -> i.contains(cleanedHost) || cleanedHost.contains("*"));
-            } else {
-                Config.globalDataMap.remove(host);
+            // 删除无用的数据
+            Set<String> wildcardKeys = Config.globalDataMap.keySet().stream()
+                    .filter(key -> key.startsWith("*."))
+                    .collect(Collectors.toSet());
+
+            Set<String> existingSuffixes = Config.globalDataMap.keySet().stream()
+                    .filter(key -> !key.startsWith("*."))
+                    .map(key -> {
+                        int dotIndex = key.indexOf(".");
+                        return dotIndex != -1 ? key.substring(dotIndex) : "";
+                    })
+                    .collect(Collectors.toSet());
+
+            Set<String> keysToRemove = wildcardKeys.stream()
+                    .filter(key -> !existingSuffixes.contains(key.substring(1)))
+                    .collect(Collectors.toSet());
+
+            keysToRemove.forEach(Config.globalDataMap::remove);
+
+            if (Config.globalDataMap.keySet().size() == 1 && Config.globalDataMap.keySet().stream().anyMatch(key -> key.equals("*"))) {
+                Config.globalDataMap.keySet().remove("*");
            }

            messageTableModel.deleteByHost(host);
--- a/src/main/java/hae/component/board/Datatable.java
+++ b/src/main/java/hae/component/board/Datatable.java
@@ -158,7 +158,7 @@ public class Datatable extends JPanel {
            @Override
            public void exportToClipboard(JComponent comp, Clipboard clip, int action) throws IllegalStateException {
                if (comp instanceof JTable) {
-                    StringSelection stringSelection = new StringSelection(getSelectedDataAtTable((JTable) comp));
+                    StringSelection stringSelection = new StringSelection(getSelectedDataAtTable((JTable) comp).replace("\0", "").replaceAll("[\\p{Cntrl}&&[^\r\n\t]]", ""));
                    clip.setContents(stringSelection, null);
                } else {
                    super.exportToClipboard(comp, clip, action);
@@ -191,13 +191,13 @@ public class Datatable extends JPanel {
            selectData.append(table.getValueAt(row, 1).toString()).append("\n");
        }

-        // 便于单行复制，去除最后一个换行符
        if (!selectData.isEmpty()) {
            selectData.deleteCharAt(selectData.length() - 1);
-            return selectData.toString();
        } else {
            return "";
        }
+
+        return selectData.toString();
    }

    public JTable getDataTable() {
--- a/src/main/java/hae/component/board/message/MessageEntry.java
+++ b/src/main/java/hae/component/board/message/MessageEntry.java
@@ -11,8 +11,10 @@ public class MessageEntry {
    private final String status;
    private final String color;
    private final String method;
+    private final String hash;
+    private final String path;

-    MessageEntry(HttpRequestResponse requestResponse, String method, String url, String comment, String length, String color, String status) {
+    MessageEntry(HttpRequestResponse requestResponse, String method, String url, String comment, String length, String color, String status, String hash, String path) {
        this.requestResponse = requestResponse;
        this.method = method;
        this.url = url;
@@ -20,6 +22,8 @@ public class MessageEntry {
        this.length = length;
        this.color = color;
        this.status = status;
+        this.hash = hash;
+        this.path = path;
    }

    public String getColor() {
@@ -49,4 +53,12 @@ public class MessageEntry {
    public HttpRequestResponse getRequestResponse() {
        return this.requestResponse;
    }
+
+    public String getHash() {
+        return this.hash;
+    }
+
+    public String getPath() {
+        return this.path;
+    }
 }
--- a/src/main/java/hae/component/board/message/MessageRenderer.java
+++ b/src/main/java/hae/component/board/message/MessageRenderer.java
@@ -4,16 +4,16 @@ import javax.swing.*;
 import javax.swing.table.DefaultTableCellRenderer;
 import java.awt.*;
 import java.util.HashMap;
-import java.util.List;
+import java.util.LinkedList;
 import java.util.Map;

 public class MessageRenderer extends DefaultTableCellRenderer {

-    private final List<MessageEntry> log;
+    private final LinkedList<MessageEntry> log;
    private final Map<String, Color> colorMap = new HashMap<>();
    private final JTable table; // 保存对表格的引用

-    public MessageRenderer(List<MessageEntry> log, JTable table) {
+    public MessageRenderer(LinkedList<MessageEntry> log, JTable table) {
        this.log = log;
        // 与BurpSuite的颜色保持一致
        this.colorMap.put("red", new Color(0xFF, 0x64, 0x64));
--- a/src/main/java/hae/component/board/message/MessageTableModel.java
+++ b/src/main/java/hae/component/board/message/MessageTableModel.java
@@ -11,6 +11,7 @@ import burp.api.montoya.ui.editor.HttpRequestEditor;
 import burp.api.montoya.ui.editor.HttpResponseEditor;
 import hae.Config;
 import hae.cache.CachePool;
+import hae.utils.project.FileProcessor;
 import hae.utils.string.HashCalculator;
 import hae.utils.string.StringProcessor;

@@ -30,16 +31,16 @@ import static burp.api.montoya.ui.editor.EditorOptions.READ_ONLY;
 public class MessageTableModel extends AbstractTableModel {
    private final MontoyaApi api;
    private final MessageTable messageTable;
-    private final JTabbedPane messageTab;
    private final JSplitPane splitPane;
-    private final List<MessageEntry> log = new ArrayList<MessageEntry>();
+    private final LinkedList<MessageEntry> log = new LinkedList<>();
    private final LinkedList<MessageEntry> filteredLog;
+    private SwingWorker<Void, Void> currentWorker;

    public MessageTableModel(MontoyaApi api) {
        this.filteredLog = new LinkedList<>();
        this.api = api;

-        messageTab = new JTabbedPane();
+        JTabbedPane messageTab = new JTabbedPane();
        UserInterface userInterface = api.userInterface();
        HttpRequestEditor requestViewer = userInterface.createHttpRequestEditor(READ_ONLY);
        HttpResponseEditor responseViewer = userInterface.createHttpResponseEditor(READ_ONLY);
@@ -92,25 +93,25 @@ public class MessageTableModel extends AbstractTableModel {
        splitPane.setRightComponent(messageTab);
    }

-    public void add(HttpRequestResponse messageInfo, String comment, String color) {
+    public void add(HttpRequestResponse messageInfo, String url, String method, String status, String length, String comment, String color, String hash, String path) {
        synchronized (log) {
-            HttpRequest httpRequest = messageInfo.request();
-            String url = httpRequest.url();
-            String method = httpRequest.method();
-
-            HttpResponse httpResponse = messageInfo.response();
-            String status = String.valueOf(httpResponse.statusCode());
-            String length = String.valueOf(httpResponse.body().length());
-
-            MessageEntry logEntry = new MessageEntry(messageInfo, method, url, comment, length, color, status);
-
-            try {
-                // 比较Hash，如若存在重复的请求或响应，则不放入消息内容里
-                byte[] reqByteA = httpRequest.toByteArray().getBytes();
-                byte[] resByteA = httpResponse.toByteArray().getBytes();
            boolean isDuplicate = false;
+            MessageEntry logEntry = new MessageEntry(messageInfo, method, url, comment, length, color, status, hash, path);

-                if (log.size() > 0) {
+            byte[] reqByteA = new byte[0];
+            byte[] resByteA = new byte[0];
+
+            if (messageInfo != null) {
+                HttpRequest httpRequest = messageInfo.request();
+                HttpResponse httpResponse = messageInfo.response();
+
+                reqByteA = httpRequest.toByteArray().getBytes();
+                resByteA = httpResponse.toByteArray().getBytes();
+            }
+
+            // 比较Hash，如若存在重复的请求或响应，则不放入消息内容里
+            try {
+                if (!log.isEmpty()) {
                    for (MessageEntry entry : log) {
                        HttpRequestResponse reqResMessage = entry.getRequestResponse();
                        byte[] reqByteB = reqResMessage.request().toByteArray().getBytes();
@@ -125,12 +126,12 @@ public class MessageTableModel extends AbstractTableModel {
                        }
                    }
                }
+            } catch (Exception ignored) {
+            }

            if (!isDuplicate) {
                log.add(logEntry);
            }
-            } catch (Exception ignored) {
-            }
        }

    }
@@ -138,11 +139,19 @@ public class MessageTableModel extends AbstractTableModel {
    public void deleteByHost(String filterText) {
        filteredLog.clear();
        List<Integer> rowsToRemove = new ArrayList<>();
+
+        if (currentWorker != null && !currentWorker.isDone()) {
+            currentWorker.cancel(true);
+        }
+
+        currentWorker = new SwingWorker<Void, Void>() {
+            @Override
+            protected Void doInBackground() {
                for (int i = 0; i < log.size(); i++) {
                    MessageEntry entry = log.get(i);
                    String host = StringProcessor.getHostByUrl(entry.getUrl());
                    if (!host.isEmpty()) {
-                if (StringProcessor.matchesHostPattern(host, filterText) || filterText.contains("*")) {
+                        if (StringProcessor.matchesHostPattern(host, filterText) || filterText.equals("*")) {
                            rowsToRemove.add(i);
                        }
                    }
@@ -153,34 +162,63 @@ public class MessageTableModel extends AbstractTableModel {
                    log.remove(row);
                }

-        if (!rowsToRemove.isEmpty()) {
-            int[] rows = rowsToRemove.stream().mapToInt(Integer::intValue).toArray();
-            fireTableRowsDeleted(rows[0], rows[rows.length - 1]);
+                return null;
            }
+        };
+
+        currentWorker.execute();
    }

    public void applyHostFilter(String filterText) {
        filteredLog.clear();
-        fireTableDataChanged();
-        String cleanedText = StringProcessor.replaceFirstOccurrence(filterText, "*.", "");

-        for (MessageEntry entry : log) {
-            String host = StringProcessor.getHostByUrl(entry.getUrl());
+        log.forEach(entry -> {
+            MessageEntry finalEntry = getEntryByFile(entry);
+            String host = StringProcessor.getHostByUrl(finalEntry.getUrl());
            if (!host.isEmpty()) {
-                if (filterText.contains("*.") && StringProcessor.matchFromEnd(StringProcessor.extractHostname(host), cleanedText)) {
-                    filteredLog.add(entry);
-                } else if (host.equals(filterText) || filterText.contains("*")) {
-                    filteredLog.add(entry);
+                if (StringProcessor.matchesHostPattern(host, filterText) || filterText.contains("*")) {
+                    filteredLog.add(finalEntry);
                }
            }
+        });
+
+        fireTableDataChanged();
+    }
+
+    private MessageEntry getEntryByFile(MessageEntry entry) {
+        HttpRequestResponse requestResponse = entry.getRequestResponse();
+        if (requestResponse == null) {
+            String url = entry.getUrl();
+            String method = entry.getMethod();
+            String status = entry.getStatus();
+            String comment = entry.getComment();
+            String color = entry.getColor();
+            String path = entry.getPath();
+            String hash = entry.getHash();
+            int length = Integer.parseInt(entry.getLength());
+
+            byte[] contents = FileProcessor.readFileContent(path, hash);
+
+            if (contents.length > length) {
+                byte[] response = Arrays.copyOf(contents, length);
+                byte[] request = Arrays.copyOfRange(contents, length, contents.length);
+                requestResponse = StringProcessor.createHttpRequestResponse(url, request, response);
+
+                int index = log.indexOf(entry);
+                entry = new MessageEntry(requestResponse, method, url, comment, String.valueOf(length), color, status, "", "");
+                log.set(index, entry);
+            }
        }

-        fireTableDataChanged();
+        return entry;
    }

    public void applyMessageFilter(String tableName, String filterText) {
        filteredLog.clear();
        for (MessageEntry entry : log) {
+            // 标志变量，表示是否满足过滤条件
+            AtomicBoolean isMatched = new AtomicBoolean(false);
+
            HttpRequestResponse requestResponse = entry.getRequestResponse();
            HttpRequest httpRequest = requestResponse.request();
            HttpResponse httpResponse = requestResponse.response();
@@ -197,9 +235,6 @@ public class MessageTableModel extends AbstractTableModel {
                    .map(HttpHeader::toString)
                    .collect(Collectors.joining("\n"));

-            // 标志变量，表示是否满足过滤条件
-            AtomicBoolean isMatched = new AtomicBoolean(false);
-
            Config.globalRules.keySet().forEach(i -> {
                for (Object[] objects : Config.globalRules.get(i)) {
                    String name = objects[1].toString();
@@ -305,7 +340,7 @@ public class MessageTableModel extends AbstractTableModel {
            if (!map2.containsKey(key)) {
                return false;
            }
-            if (!areInnerMapsEqual(map1.get(key), map2.get(key))) {
+            if (areInnerMapsEqual(map1.get(key), map2.get(key))) {
                return false;
            }
        }
@@ -315,29 +350,28 @@ public class MessageTableModel extends AbstractTableModel {

    private boolean areInnerMapsEqual(Map<String, Object> innerMap1, Map<String, Object> innerMap2) {
        if (innerMap1.size() != innerMap2.size()) {
-            return false;
+            return true;
        }

        for (String key : innerMap1.keySet()) {
            if (!innerMap2.containsKey(key)) {
-                return false;
+                return true;
            }
            Object value1 = innerMap1.get(key);
            Object value2 = innerMap2.get(key);

            // 如果值是Map，则递归对比
            if (value1 instanceof Map && value2 instanceof Map) {
-                if (!areInnerMapsEqual((Map<String, Object>) value1, (Map<String, Object>) value2)) {
-                    return false;
-                }
-            } else if (!value1.equals(value2)) {
-                return false;
-            }
-        }
-
+                if (areInnerMapsEqual((Map<String, Object>) value1, (Map<String, Object>) value2)) {
                    return true;
                }
+            } else if (!value1.equals(value2)) {
+                return true;
+            }
+        }

+        return false;
+    }

    public JSplitPane getSplitPane() {
        return splitPane;
@@ -347,7 +381,7 @@ public class MessageTableModel extends AbstractTableModel {
        return messageTable;
    }

-    public List<MessageEntry> getLogs() {
+    public LinkedList<MessageEntry> getLogs() {
        return log;
    }

@@ -363,11 +397,11 @@ public class MessageTableModel extends AbstractTableModel {

    @Override
    public Object getValueAt(int rowIndex, int columnIndex) {
-        if (filteredLog.isEmpty()) {
-            return "";
-        }
+        if (!filteredLog.isEmpty()) {
+            try {
                MessageEntry messageEntry = filteredLog.get(rowIndex);

+                if (messageEntry != null) {
                    return switch (columnIndex) {
                        case 0 -> messageEntry.getMethod();
                        case 1 -> messageEntry.getUrl();
@@ -378,6 +412,13 @@ public class MessageTableModel extends AbstractTableModel {
                        default -> "";
                    };
                }
+            } catch (Exception e) {
+                api.logging().logToError("getValueAt: " + e.getMessage());
+            }
+        }
+
+        return "";
+    }

    @Override
    public String getColumnName(int columnIndex) {
@@ -393,10 +434,8 @@ public class MessageTableModel extends AbstractTableModel {
    }

    public class MessageTable extends JTable {
-        private MessageEntry MessageEntry;
+        private MessageEntry messageEntry;
        private SwingWorker<Object, Void> currentWorker;
-        // 设置响应报文返回的最大长度
-        private final int MAX_LENGTH = 5242880;
        private int lastSelectedIndex = -1;
        private final HttpRequestEditor requestEditor;
        private final HttpResponseEditor responseEditor;
@@ -410,10 +449,6 @@ public class MessageTableModel extends AbstractTableModel {
        @Override
        public void changeSelection(int row, int col, boolean toggle, boolean extend) {
            super.changeSelection(row, col, toggle, extend);
-            int selectedIndex = convertRowIndexToModel(row);
-            if (lastSelectedIndex != selectedIndex) {
-                lastSelectedIndex = selectedIndex;
-                MessageEntry = filteredLog.get(selectedIndex);

            requestEditor.setRequest(HttpRequest.httpRequest("Loading..."));
            responseEditor.setResponse(HttpResponse.httpResponse("Loading..."));
@@ -424,32 +459,26 @@ public class MessageTableModel extends AbstractTableModel {

            currentWorker = new SwingWorker<>() {
                @Override
-                    protected ByteArray[] doInBackground() {
-                        ByteArray requestByte = MessageEntry.getRequestResponse().request().toByteArray();
-                        ByteArray responseByte = MessageEntry.getRequestResponse().response().toByteArray();
+                protected Void doInBackground() {
+                    int selectedIndex = convertRowIndexToModel(row);
+                    if (lastSelectedIndex != selectedIndex) {
+                        lastSelectedIndex = selectedIndex;
+                        messageEntry = filteredLog.get(selectedIndex);
+
+                        HttpRequestResponse httpRequestResponse = messageEntry.getRequestResponse();
+
+                        ByteArray requestByte = httpRequestResponse.request().toByteArray();
+                        ByteArray responseByte = httpRequestResponse.response().toByteArray();
+
+                        requestEditor.setRequest(HttpRequest.httpRequest(messageEntry.getRequestResponse().httpService(), requestByte));
+                        responseEditor.setResponse(HttpResponse.httpResponse(responseByte));

-                        if (responseByte.length() > MAX_LENGTH) {
-                            String ellipsis = "\r\n......";
-                            responseByte = responseByte.subArray(0, MAX_LENGTH).withAppended(ellipsis);
                    }

-                        return new ByteArray[]{requestByte, responseByte};
-                    }
-
-                    @Override
-                    protected void done() {
-                        if (!isCancelled()) {
-                            try {
-                                ByteArray[] result = (ByteArray[]) get();
-                                requestEditor.setRequest(HttpRequest.httpRequest(MessageEntry.getRequestResponse().httpService(), result[0]));
-                                responseEditor.setResponse(HttpResponse.httpResponse(result[1]));
-                            } catch (Exception ignored) {
-                            }
-                        }
+                    return null;
                }
            };
            currentWorker.execute();
        }
    }
 }
-}
--- a/src/main/java/hae/component/config/Config.java
+++ b/src/main/java/hae/component/config/Config.java
@@ -250,7 +250,7 @@ public class Config extends JPanel {

    private void onlineUpdateActionPerformed(ActionEvent e) {
        // 添加提示框防止用户误触导致配置更新
-        int retCode = JOptionPane.showConfirmDialog(null, "Do you want to update rules?", "Info", JOptionPane.YES_NO_OPTION);
+        int retCode = JOptionPane.showConfirmDialog(this, "Do you want to update rules?", "Info", JOptionPane.YES_NO_OPTION);
        if (retCode == JOptionPane.YES_OPTION) {
            configLoader.initRulesByNet();
            reloadActionPerformed(null);
--- a/src/main/java/hae/component/rule/Rule.java
+++ b/src/main/java/hae/component/rule/Rule.java
@@ -97,7 +97,7 @@ public class Rule extends JPanel {
        Display ruleDisplay = new Display();
        ruleDisplay.formatTextField.setText("{0}");

-        int showState = JOptionPane.showConfirmDialog(null, ruleDisplay, "Add Rule", JOptionPane.OK_OPTION);
+        int showState = JOptionPane.showConfirmDialog(this, ruleDisplay, "Add Rule", JOptionPane.OK_OPTION);
        if (showState == YES_OPTION) {
            Vector<Object> ruleData = new Vector<>();
            ruleData.add(false);
@@ -132,7 +132,7 @@ public class Rule extends JPanel {

            ruleDisplay.formatTextField.setEnabled(ruleDisplay.engineComboBox.getSelectedItem().toString().equals("nfa"));

-            int showState = JOptionPane.showConfirmDialog(null, ruleDisplay, "Edit Rule", JOptionPane.OK_OPTION);
+            int showState = JOptionPane.showConfirmDialog(this, ruleDisplay, "Edit Rule", JOptionPane.OK_OPTION);
            if (showState == 0) {
                int select = ruleTable.convertRowIndexToModel(ruleTable.getSelectedRow());
                model.setValueAt(ruleDisplay.ruleNameTextField.getText(), select, 1);
@@ -151,7 +151,7 @@ public class Rule extends JPanel {

    private void ruleRemoveActionPerformed(ActionEvent e, JTable ruleTable, JTabbedPane tabbedPane) {
        if (ruleTable.getSelectedRowCount() >= 1) {
-            if (JOptionPane.showConfirmDialog(null, "Are you sure you want to remove this rule?", "Info", JOptionPane.YES_NO_OPTION) == 0) {
+            if (JOptionPane.showConfirmDialog(this, "Are you sure you want to remove this rule?", "Info", JOptionPane.YES_NO_OPTION) == 0) {
                DefaultTableModel model = (DefaultTableModel) ruleTable.getModel();
                int select = ruleTable.convertRowIndexToModel(ruleTable.getSelectedRow());

--- a/src/main/java/hae/component/rule/Rules.java
+++ b/src/main/java/hae/component/rule/Rules.java
@@ -109,7 +109,7 @@ public class Rules extends JTabbedPane {

    private void deleteRuleGroupActionPerformed(ActionEvent e) {
        if (getTabCount() > 2) {
-            int retCode = JOptionPane.showConfirmDialog(null, "Do you want to delete this rule group?", "Info",
+            int retCode = JOptionPane.showConfirmDialog(this, "Do you want to delete this rule group?", "Info",
                    JOptionPane.YES_NO_OPTION);
            if (retCode == JOptionPane.YES_OPTION) {
                String title = getTitleAt(getSelectedIndex());
--- a/src/main/java/hae/instances/editor/RequestEditor.java
+++ b/src/main/java/hae/instances/editor/RequestEditor.java
@@ -122,7 +122,9 @@ public class RequestEditor implements HttpRequestEditorProvider {
        boolean isBlockHost = false;
        for (String hostName : hostList) {
            String cleanedHost = StringProcessor.replaceFirstOccurrence(hostName, "*.", "");
-            if (StringProcessor.matchFromEnd(host, cleanedHost)) {
+            if (hostName.contains("*.") && StringProcessor.matchFromEnd(host, cleanedHost)) {
+                isBlockHost = true;
+            } else if (host.equals(hostName) || hostName.equals("*")) {
                isBlockHost = true;
            }
        }
--- a/src/main/java/hae/instances/http/HttpMessageHandler.java
+++ b/src/main/java/hae/instances/http/HttpMessageHandler.java
@@ -12,6 +12,7 @@ import hae.instances.http.utils.MessageProcessor;
 import hae.utils.ConfigLoader;
 import hae.utils.string.StringProcessor;

+import javax.swing.*;
 import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.List;
@@ -45,6 +46,7 @@ public class HttpMessageHandler implements HttpHandler {

        Annotations annotations = httpRequestToBeSent.annotations();

+        try {
            httpRequest.set(httpRequestToBeSent);

            host.set(StringProcessor.getHostByUrl(httpRequestToBeSent.url()));
@@ -59,6 +61,9 @@ public class HttpMessageHandler implements HttpHandler {
                List<Map<String, String>> result = messageProcessor.processRequest(host.get(), httpRequestToBeSent, true);
                setColorAndCommentList(result);
            }
+        } catch (Exception e) {
+            api.logging().logToError("handleHttpRequestToBeSent: " + e.getMessage());
+        }

        return RequestToBeSentAction.continueWith(httpRequestToBeSent, annotations);
    }
@@ -80,7 +85,19 @@ public class HttpMessageHandler implements HttpHandler {
                HttpRequestResponse httpRequestResponse = HttpRequestResponse.httpRequestResponse(httpRequest.get(), httpResponseReceived);

                // 添加到Databoard
-                messageTableModel.add(httpRequestResponse, comment, color);
+                String method = httpRequest.get().method();
+                String url = httpRequest.get().url();
+                String status = String.valueOf(httpResponseReceived.statusCode());
+                String length = String.valueOf(httpResponseReceived.toByteArray().length());
+
+                // 后台提交，防止线程阻塞
+                new SwingWorker<Void, Void>() {
+                    @Override
+                    protected Void doInBackground() {
+                        messageTableModel.add(httpRequestResponse, url, method, status, length, comment, color, "", "");
+                        return null;
+                    }
+                }.run();
            }
        }

@@ -88,7 +105,7 @@ public class HttpMessageHandler implements HttpHandler {
    }

    private void setColorAndCommentList(List<Map<String, String>> result) {
-        if (result != null && !result.isEmpty() && result.size() > 0) {
+        if (result != null && !result.isEmpty()) {
            colorList.get().add(result.get(0).get("color"));
            commentList.get().add(result.get(1).get("comment"));
        }
--- a/src/main/java/hae/instances/http/utils/MessageProcessor.java
+++ b/src/main/java/hae/instances/http/utils/MessageProcessor.java
@@ -71,7 +71,7 @@ public class MessageProcessor {
        List<Map<String, String>> highlightList = new ArrayList<>();
        List<Map<String, String>> extractList = new ArrayList<>();

-        if (obj != null && !obj.isEmpty() && obj.size() > 0) {
+        if (obj != null && !obj.isEmpty()) {
            if (actionFlag) {
                List<List<String>> resultList = extractColorsAndComments(obj);
                List<String> colorList = resultList.get(0);
--- a/src/main/java/hae/instances/http/utils/RegularMatcher.java
+++ b/src/main/java/hae/instances/http/utils/RegularMatcher.java
@@ -126,7 +126,7 @@ public class RegularMatcher {
            String[] splitHost = host.split("\\.");
            String onlyHost = host.split(":")[0];

-            String anyHost = (splitHost.length > 2 && !onlyHost.matches("\\b(?:\\d{1,3}\\.){3}\\d{1,3}\\b")) ? StringProcessor.replaceFirstOccurrence(onlyHost, splitHost[0], "*") : "";
+            String anyHost = (splitHost.length > 2 && !StringProcessor.matchHostIsIp(onlyHost)) ? StringProcessor.replaceFirstOccurrence(onlyHost, splitHost[0], "*") : "";

            if (!Config.globalDataMap.containsKey(anyHost) && anyHost.length() > 0) {
                // 添加通配符Host，实际数据从查询哪里将所有数据提取
--- a/src/main/java/hae/utils/project/FileProcessor.java
+++ b/src/main/java/hae/utils/project/FileProcessor.java
@@ -0,0 +1,47 @@
+package hae.utils.project;
+
+import java.io.File;
+import java.nio.file.DirectoryStream;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Comparator;
+
+public class FileProcessor {
+    public static void deleteDirectoryWithContents(Path pathToBeDeleted) {
+        if (pathToBeDeleted != null) {
+            try {
+                Files.walk(pathToBeDeleted)
+                        .sorted(Comparator.reverseOrder())
+                        .map(Path::toFile)
+                        .forEach(File::delete);
+            } catch (Exception ignored) {
+            }
+        }
+    }
+
+    public static byte[] readFileContent(String basePath, String fileName) {
+        Path filePath = Paths.get(basePath, fileName);
+        Path path = Paths.get(basePath);
+        try {
+            byte[] fileContent = Files.readAllBytes(filePath);
+
+            Files.deleteIfExists(filePath);
+
+            boolean isEmpty = isDirectoryEmpty(path);
+            if (isEmpty) {
+                Files.deleteIfExists(path);
+            }
+
+            return fileContent;
+        } catch (Exception e) {
+            return new byte[0];
+        }
+    }
+
+    private static boolean isDirectoryEmpty(Path directory) throws Exception {
+        try (DirectoryStream<Path> dirStream = Files.newDirectoryStream(directory)) {
+            return !dirStream.iterator().hasNext();
+        }
+    }
+}
--- a/src/main/java/hae/utils/project/ProjectProcessor.java
+++ b/src/main/java/hae/utils/project/ProjectProcessor.java
@@ -2,14 +2,20 @@ package hae.utils.project;

 import burp.api.montoya.MontoyaApi;
 import hae.utils.project.model.HaeFileContent;
+import org.yaml.snakeyaml.LoaderOptions;
 import org.yaml.snakeyaml.Yaml;

 import java.io.*;
 import java.nio.charset.StandardCharsets;
-import java.util.List;
-import java.util.Map;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.StandardCopyOption;
+import java.util.*;
+import java.util.concurrent.Callable;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
 import java.util.zip.ZipEntry;
-import java.util.zip.ZipInputStream;
+import java.util.zip.ZipFile;
 import java.util.zip.ZipOutputStream;

 public class ProjectProcessor {
@@ -19,59 +25,163 @@ public class ProjectProcessor {
        this.api = api;
    }

-    public boolean createHaeFile(String haeFilePath, String host, Map<String, List<String>> dataMap, Map<String, Map<String, String>> httpMap) {
+    public boolean createHaeFile(String haeFilePath, String host, Map<String, List<String>> dataMap, Map<String, Map<String, Object>> urlMap, Map<String, Map<String, Object>> httpMap) {
+        ExecutorService executor = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors() * 2);
+
+        List<Callable<Void>> tasks = new ArrayList<>();
+
        ByteArrayOutputStream dataYamlStream = new ByteArrayOutputStream();
-        ByteArrayOutputStream httpYamlStream = new ByteArrayOutputStream();
+        ByteArrayOutputStream urlYamlStream = new ByteArrayOutputStream();
        Yaml yaml = new Yaml();

        yaml.dump(dataMap, new OutputStreamWriter(dataYamlStream, StandardCharsets.UTF_8));
-        yaml.dump(httpMap, new OutputStreamWriter(httpYamlStream, StandardCharsets.UTF_8));
+        yaml.dump(urlMap, new OutputStreamWriter(urlYamlStream, StandardCharsets.UTF_8));

        try (ZipOutputStream zipOut = new ZipOutputStream(new FileOutputStream(haeFilePath))) {
            zipOut.putNextEntry(new ZipEntry("info"));
            zipOut.write(host.getBytes(StandardCharsets.UTF_8));
            zipOut.closeEntry();

-            zipOut.putNextEntry(new ZipEntry("data.yml"));
+            zipOut.putNextEntry(new ZipEntry("data"));
            zipOut.write(dataYamlStream.toByteArray());
            zipOut.closeEntry();

-            zipOut.putNextEntry(new ZipEntry("http.yml"));
-            zipOut.write(httpYamlStream.toByteArray());
+            zipOut.putNextEntry(new ZipEntry("url"));
+            zipOut.write(urlYamlStream.toByteArray());
            zipOut.closeEntry();
+
+            for (String httpHash : httpMap.keySet()) {
+                Map<String, Object> httpItem = httpMap.get(httpHash);
+                tasks.add(() -> {
+                    try {
+                        ByteArrayOutputStream httpOutStream = new ByteArrayOutputStream();
+                        byte[] request = (byte[]) httpItem.get("request");
+                        byte[] response = (byte[]) httpItem.get("response");
+
+                        httpOutStream.write(response);
+                        httpOutStream.write(request);
+
+                        synchronized (zipOut) {
+                            zipOut.putNextEntry(new ZipEntry(String.format("http/%s", httpHash)));
+                            zipOut.write(httpOutStream.toByteArray());
+                            zipOut.closeEntry();
+                        }
                    } catch (Exception e) {
-            api.logging().logToOutput(e.getMessage());
+                        api.logging().logToError("createHaeFile: " + e.getMessage());
+                    }
+
+                    return null;
+                });
+            }
+
+            executor.invokeAll(tasks);
+        } catch (Exception e) {
+            api.logging().logToError("createHaeFile: " + e.getMessage());
            return false;
+        } finally {
+            executor.shutdown();
        }

        return true;
    }

    public HaeFileContent readHaeFile(String haeFilePath) {
-        HaeFileContent haeFileContent = new HaeFileContent(api);
-        Yaml yaml = new Yaml();
+        ExecutorService executor = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors() * 2);
+        List<Callable<Void>> tasks = new ArrayList<>();

-        try (ZipInputStream zipIn = new ZipInputStream(new FileInputStream(haeFilePath))) {
-            ZipEntry entry;
-            while ((entry = zipIn.getNextEntry()) != null) {
-                switch (entry.getName()) {
-                    case "info":
-                        haeFileContent.setHost(new String(zipIn.readAllBytes(), StandardCharsets.UTF_8));
-                        break;
-                    case "data.yml":
-                        haeFileContent.setDataMap(yaml.load(new InputStreamReader(zipIn, StandardCharsets.UTF_8)));
-                        break;
-                    case "http.yml":
-                        haeFileContent.setHttpMap(yaml.load(new InputStreamReader(zipIn, StandardCharsets.UTF_8)));
-                        break;
+        HaeFileContent haeFileContent = new HaeFileContent(api);
+        LoaderOptions loaderOptions = new LoaderOptions();
+        loaderOptions.setMaxAliasesForCollections(Integer.MAX_VALUE);
+        loaderOptions.setCodePointLimit(Integer.MAX_VALUE);
+        Yaml yaml = new Yaml(loaderOptions);
+        Path tempDirectory = null;
+
+        try {
+            if (hasValidStructure(haeFilePath)) {
+                tempDirectory = Files.createTempDirectory("hae");
+                haeFileContent.setHttpPath(tempDirectory.toString());
+
+                try (ZipFile zipFile = new ZipFile(haeFilePath)) {
+                    Enumeration<? extends ZipEntry> entries = zipFile.entries();
+                    while (entries.hasMoreElements()) {
+                        ZipEntry entry = entries.nextElement();
+                        String fileName = entry.getName();
+                        if (fileName.startsWith("http/")) {
+                            Path filePath = tempDirectory.resolve(fileName.substring("http/".length()));
+
+                            tasks.add(() -> {
+                                try (InputStream in = zipFile.getInputStream(entry)) {
+                                    Files.copy(in, filePath, StandardCopyOption.REPLACE_EXISTING);
+                                } catch (IOException e) {
+                                    api.logging().logToError("readHaeFile: " + e.getMessage());
+                                }
+
+                                return null;
+                            });
+                        } else {
+                            try (InputStream in = zipFile.getInputStream(entry)) {
+                                switch (fileName) {
+                                    case "info" ->
+                                            haeFileContent.setHost(new String(in.readAllBytes(), StandardCharsets.UTF_8));
+                                    case "data" ->
+                                            haeFileContent.setDataMap(yaml.load(new InputStreamReader(in, StandardCharsets.UTF_8)));
+                                    case "url" ->
+                                            haeFileContent.setUrlMap(yaml.load(new InputStreamReader(in, StandardCharsets.UTF_8)));
+                                }
+                            }
+                        }
+                    }
+
+                    executor.invokeAll(tasks);
                }
-                zipIn.closeEntry();
            }
        } catch (Exception e) {
-            api.logging().logToOutput(e.getMessage());
-            return null;
+            api.logging().logToError("readHaeFile: " + e.getMessage());
+            if (tempDirectory != null) {
+                FileProcessor.deleteDirectoryWithContents(tempDirectory);
            }
+            haeFileContent = null;
+        } finally {
+            executor.shutdown();
+        }
+
        return haeFileContent;
    }
+
+    private boolean hasValidStructure(String zipFilePath) {
+        Set<String> requiredRootEntries = new HashSet<>();
+        requiredRootEntries.add("info");
+        requiredRootEntries.add("data");
+        requiredRootEntries.add("url");
+
+        boolean hasHttpDirectoryWithFiles = false;
+
+        try {
+            ZipFile zipFile = new ZipFile(zipFilePath);
+            Enumeration<? extends ZipEntry> entries = zipFile.entries();
+
+            while (entries.hasMoreElements()) {
+                ZipEntry entry = entries.nextElement();
+                String name = entry.getName();
+
+                if (!entry.isDirectory() && !name.contains("/")) {
+                    requiredRootEntries.remove(name);
+                }
+
+                if (name.startsWith("http/") && !entry.isDirectory()) {
+                    hasHttpDirectoryWithFiles = true;
+                }
+
+                if (requiredRootEntries.isEmpty() && hasHttpDirectoryWithFiles) {
+                    break;
+                }
+            }
+
+            zipFile.close();
+        } catch (Exception ignored) {
+        }
+
+        return requiredRootEntries.isEmpty() && hasHttpDirectoryWithFiles;
+    }
 }

--- a/src/main/java/hae/utils/project/model/HaeFileContent.java
+++ b/src/main/java/hae/utils/project/model/HaeFileContent.java
@@ -11,13 +11,14 @@ import java.util.Map;
 public class HaeFileContent {
    private final MontoyaApi api;
    private String host;
+    private String httpPath;
    private final Map<String, List<String>> dataMap;
-    private final Map<String, Map<String, String>> httpMap;
+    private final Map<String, Map<String, String>> urlMap;

    public HaeFileContent(MontoyaApi api) {
        this.api = api;
        this.dataMap = new HashMap<>();
-        this.httpMap = new HashMap<>();
+        this.urlMap = new HashMap<>();
    }

    public String getHost() {
@@ -28,14 +29,22 @@ public class HaeFileContent {
        return dataMap;
    }

-    public Map<String, Map<String, String>> getHttpMap() {
-        return httpMap;
+    public Map<String, Map<String, String>> getUrlMap() {
+        return urlMap;
+    }
+
+    public String getHttpPath() {
+        return httpPath;
    }

    public void setHost(String host) {
        this.host = host;
    }

+    public void setHttpPath(String path) {
+        this.httpPath = path;
+    }
+
    public void setDataMap(Map<String, List<Object>> dataMap) {
        for (Map.Entry<String, List<Object>> entry : dataMap.entrySet()) {
            List<String> values = new ArrayList<>();
@@ -50,8 +59,8 @@ public class HaeFileContent {
        }
    }

-    public void setHttpMap(Map<String, Map<String, Object>> httpMap) {
-        for (Map.Entry<String, Map<String, Object>> entry : httpMap.entrySet()) {
+    public void setUrlMap(Map<String, Map<String, Object>> urlMap) {
+        for (Map.Entry<String, Map<String, Object>> entry : urlMap.entrySet()) {
            Map<String, String> newValues = new HashMap<>();
            Map<String, Object> values = entry.getValue();
            for (String key : values.keySet()) {
@@ -61,7 +70,7 @@ public class HaeFileContent {
                    newValues.put(key, values.get(key).toString());
                }
            }
-            this.httpMap.put(entry.getKey(), newValues);
+            this.urlMap.put(entry.getKey(), newValues);
        }
    }
 }
--- a/src/main/java/hae/utils/string/StringProcessor.java
+++ b/src/main/java/hae/utils/string/StringProcessor.java
@@ -1,8 +1,17 @@
 package hae.utils.string;

+import burp.api.montoya.core.ByteArray;
+import burp.api.montoya.http.HttpService;
+import burp.api.montoya.http.message.HttpRequestResponse;
+import burp.api.montoya.http.message.requests.HttpRequest;
+import burp.api.montoya.http.message.responses.HttpResponse;
+
 import java.net.URL;
+import java.time.LocalDateTime;
+import java.time.format.DateTimeFormatter;
 import java.util.HashMap;
 import java.util.Map;
+import java.util.UUID;

 public class StringProcessor {
    public static String replaceFirstOccurrence(String original, String find, String replace) {
@@ -55,6 +64,24 @@ public class StringProcessor {
        return matchesDirectly || matchesPattern;
    }

+    public static HttpRequestResponse createHttpRequestResponse(String url, byte[] request, byte[] response) {
+        HttpService httpService = HttpService.httpService(url);
+        HttpRequest httpRequest = HttpRequest.httpRequest(httpService, ByteArray.byteArray(request));
+        HttpResponse httpResponse = HttpResponse.httpResponse(ByteArray.byteArray(response));
+        return HttpRequestResponse.httpRequestResponse(httpRequest, httpResponse);
+    }
+
+    public static String getCurrentTime() {
+        DateTimeFormatter formatter = DateTimeFormatter.ofPattern("yyyyMMdd_HHmmss");
+        LocalDateTime now = LocalDateTime.now();
+        return now.format(formatter);
+    }
+
+    public static String getRandomUUID() {
+        UUID uuid = UUID.randomUUID();
+        return uuid.toString();
+    }
+
    public static String mergeComment(String comment) {
        if (!comment.contains(",")) {
            return comment;
@@ -92,6 +119,10 @@ public class StringProcessor {
        return host;
    }

+    public static boolean matchHostIsIp(String host) {
+        return host.matches("\\b(?:\\d{1,3}\\.){3}\\d{1,3}\\b");
+    }
+
    private static Map<String, Integer> getStringIntegerMap(String comment) {
        Map<String, Integer> itemCounts = new HashMap<>();
        String[] items = comment.split(", ");
Author	SHA1	Message	Date
gh0stkey	3a8d9eae11	Version: 3.2.2 Update	2024-06-19 22:20:46 +08:00
gh0stkey	e5f55b6c4c	Version: 3.2.2 Update	2024-06-19 22:16:57 +08:00
gh0stkey	54973d9f4f	Version: 3.2.1 Update	2024-05-30 16:01:25 +08:00
gh0stkey	fb347a8dc6	Version: 3.2.1 Update	2024-05-30 15:56:49 +08:00
gh0stkey	04b6652b03	Version: 3.2.1 Update	2024-05-30 14:42:26 +08:00
gh0stkey	6d4abae898	Version: 3.2.1 Update	2024-05-30 14:37:01 +08:00
gh0stkey	97172fab45	Update	2024-05-26 15:09:12 +08:00
EvilChen	ba3b206acf	Update build.gradle	2024-05-24 17:13:31 +08:00