如何使用WebStor快速检查你组织网络中的所有网站相关安全技术

关于WebStor

WebStor是一款功能强大的网站安全检测工具脚本，在WebStor的帮助下，广大研究人员可以轻松快速枚举当前组织中的全部网站，以及响应存储、数据存储和其他所使用的已知Web技术。除此之外，WebStor还可以识别相关的0 day漏洞以及利用技术。

快速识别组织中易受攻击的Web技术

WebStor基于Python语言开发实现，可以实现快速枚举组织整个网络中所有的网站。因为很多情况下，广大安全研究人员几乎不可能跟踪分布在不同单元和网络上的不同管理员部署的所有Web技术，因此WebStor尤其适合解决这类具有分散管理的中大型组织中出现的独特问题。

WebStor可以通过执行下列操作任务来实现其目标：

• 执行DNS区域传输来收集组织网络中的A记录和CNAME记录。
• 使用Masscan扫描组织网络范围中开放的HTTP/HTTPS端口，以及组织网络的A记录和CNAME记录中存在的那些组织地址范围外的任何IP地址。
• 使用Python的requests库收集全部的响应信息并存储在MariaDB数据库中。除了IP地址之外，与开放HTTP/HTTPS端口的IP相对应的所有DNS名称都将包含在请求中，以便目标网站在使用不同Header的时候不会导致遗漏任何站点。
• 下载WappalyzerWeb技术数据库并存储在MariaDB数据库中，使用户能够按名称查询常见Web技术位置。
• 允许用户查询包含了自定义正则表达式的存储响应位置。

支持的平台

当前版本的WebStor支持在Linux系统上运行，由于该工具基于Python开发，因此也可以轻松移植到Windows系统上使用。

工具架构

工具要求

应用程序

• Masscan：如果你想要使用Cron作业来更新数据库，则必须在通过WebStor执行Masscan扫描的时候提供sudo密码。
• MariaDB 10.0.5或更高版本：WebStor尝试的默认凭据将是root和空密码。

Python库

pip3 install dnspython

pip3 install beautifulsoup4

pip3 install mysql-connector-python

pip3 install js-regex

pip3 install gevent

pip3 install requests

工具安装

首先，我们需要使用下列命令将该项目源码克隆至本地：

git clone https://github.com/RossGeerlings/webstor.git

如果你想直接运行WebStor而不需要进行自定义配置的话，可以直接安装好该工具的依赖组件，然后运行下列命令来安装WebStor：

sudo pip3 install webstor

如果通过PyPI安装的话，WebStor将会直接配置到系统环境变量中，我们可以直接使用命令行工具切换到WebStor的工作目录下，然后运行下列命令来启动WebStor（不需要执行“webstor.py”）：

webstor

或

webstor -g

工具基础使用

webstor.py [-h] [--ADD-HTTP-PORT HTTPPORTTOADD] [--CLEAR-HTTP]

                  [--ADD-HTTPS-PORT HTTPSPORTTOADD] [--CLEAR-HTTPS]

                  [--ADD-CUSTOM-FINGERPRINT FINGERPRINT]

                  [--DELETE-CUSTOM-FINGERPRINT FINGERPRINTNAMETODELETE]

                  [--IMPORT-CUSTOM-FINGERPRINT IMPORTFINGERPRINTFILE]

                  [--CLEAR-CUSTOM-FINGERPRINTS] [--SHOW-CONFIG]

                  [--SHOW-CONFIG-FULL] [--RUN-MASSCAN]

                  [--SET-MASSCAN-RANGES SETSCANRANGES]

                  [--ADD-RANGE RANGETOADD] [--DELETE-RANGE RANGETODELETE]

                  [--IMPORT-MASSCAN-RANGES IMPORTSCANRANGES]

                  [--ADD-PATH PATHTOADD] [--DELETE-PATH PATHTODELETE]

                  [--CLEAR-PATHS] [--REFRESH-RESPONSES]

                  [--RESPONSES-ADD-FOR-PATH RESPONSESADDFORPATH]

                  [--SEARCH-PATTERN SEARCHPATTERN]

                  [--SEARCH-CUSTOM-FINGERPRINT SEARCHFINGERPRINT]

                  [--SEARCH-WAPPALYZER SEARCHWAPPALYZER] [--NO-TSIG-KEY]

                  [--TSIG-KEY-IMPORT IMPORTTSIGFILE]

                  [--TSIG-KEY-REP*LACE REP*LACEMENTTSIGFILE]

                  [--DELETE-TSIG TSIGTODELETE]

                  [--USE-TSIG-FILE-ONLY USETSIGFILEONLY]

                  [--DOWNLOAD-NEW-WAPPALYZER] [--LIST-WAPPALYZER-TECH-NAMES]

                  [--ZONE-XFER] [--ADD-DOMAIN DOMAINDETAILS]

                  [--DELETE-DOMAIN DOMAINTODELETE]

                  [--IMPORT-ZONE-FILE IMPORTZONEFILE] [--CLEAR-DOMAINS]

                  [--LIST-DOMAINS] [--LIST-OUTSIDE] [--SQL-CREDS SQLCREDSFILE]

 

optional arguments:

  -h, --help            show this help message and exit

  --ADD-HTTP-PORT HTTPPORTTOADD, -a HTTPPORTTOADD

                        Add a custom HTTP port.

  --CLEAR-HTTP, -aC     Clear any custom HTTP ports and revert to default of

                        80.

  --ADD-HTTPS-PORT HTTPSPORTTOADD, -b HTTPSPORTTOADD

                        Add a custom HTTPS port.

  --CLEAR-HTTPS, -bC    Clear any custom HTTPS ports and revert to default of

                        443.

  --ADD-CUSTOM-FINGERPRINT FINGERPRINT, -c FINGERPRINT

                        Add a custom fingerprint in the form <Name>,<RegEx>.

  --DELETE-CUSTOM-FINGERPRINT FINGERPRINTNAMETODELETE, -cD FINGERPRINTNAMETODELETE

                        Delete a custom fingerprint by name.

  --IMPORT-CUSTOM-FINGERPRINT IMPORTFINGERPRINTFILE, -cI IMPORTFINGERPRINTFILE

                        Import a custom fingerprint file with the path

                        specified.

  --CLEAR-CUSTOM-FINGERPRINTS, -cC

                        Clears all custom fingerprints stored in DB.

  --SHOW-CONFIG, -g     Show current WebStor configuration (brief).

  --SHOW-CONFIG-FULL, -gF

                        Show current WebStor configuration (full).

  --RUN-MASSCAN, -m     Runs a new port scan with Masscan on all configured

                        TCP ports for HTTP and HTTPS, against all configured

                        ranges and any IP addresses from DNS records that are

                        outside those ranges.

  --SET-MASSCAN-RANGES SETSCANRANGES, -mR SETSCANRANGES

                        Scan range or ranges, rep*laces existing ranges in DB,

                        comma separated, such as: -s

                        10.10.0.0/16,10.13.0.0/16,192.168.1.0/24

  --IMPORT-MASSCAN-RANGES IMPORTSCANRANGES, -mI IMPORTSCANRANGES

                        Import scan ranges (CIDR blocks) from a specified

                        file.

  --DELETE-RANGE RANGETODELETE, -mD RANGETODELETE

                        Delete scan range.

  --ADD-PATH PATHTOADD, -p PATHTOADD

                        Add paths for which to request and store responses

                        besides '/'.

  --DELETE-PATH PATHTODELETE, -pD PATHTODELETE

                        Delete paths for which to request and store responses

                        besides '/'.

  --CLEAR-PATHS, -pC    Clear any custom URL request paths and revert to

                        default of '/'.

  --REFRESH-RESPONSES, -r

                        Refresh URL responses in DB.

  --RESPONSES-ADD-FOR-PATH RESPONSESADDFORPATH, -rP RESPONSESADDFORPATH

                        Add URL responses for a one-time path in with the

                        current responses in the DB.

  --SEARCH-PATTERN SEARCHPATTERN, -sP SEARCHPATTERN

                        Search for string or regular expression in WebStor

                        database.

  --SEARCH-CUSTOM-FINGERPRINT SEARCHFINGERPRINT, -sC SEARCHFINGERPRINT

                        Search for technology by name of user-provided custom

                        fingerprint.

  --SEARCH-WAPPALYZER SEARCHWAPPALYZER, -sW SEARCHWAPPALYZER

                        Search for technology by name (from Wappalyzer Tech

DB) in WebStor DB.

  --NO-TSIG-KEY, -tN    Do not use DNSSec TSIG key stored in database or a

                        file, even if present.

  --TSIG-KEY-IMPORT IMPORTTSIGFILE, -tI IMPORTTSIGFILE

                        Import a specified TSIG key file into the database

  --TSIG-KEY-REP*LACE REP*LACEMENTTSIGFILE, -tR REP*LACEMENTTSIGFILE

                        Rep*lace a TSIG key in the database with a specified

                        file

  --DELETE-TSIG TSIGTODELETE, -dT TSIGTODELETE

                        Delete a TSIG key from the database by name.

  --USE-TSIG-FILE-ONLY USETSIGFILEONLY, -tF USETSIGFILEONLY

                        Only use tsig file specified (full path), do not use

                        TSIGs stored in the DB. Applies to all domains,

                        limiting WebStor to one TSIG for zone transfers in the

                        current execution.

  --DOWNLOAD-NEW-WAPPALYZER, -w

                        Download a new Wappalyzer fingerprints file directly

                        from GitHub. Overwrites existing Wappalyzer

                        fingerprint data.

  --LIST-WAPPALYZER-TECH-NAMES, -wL

                        List the names of all Wappalyzer technologies in the

                        database.

  --ZONE-XFER, -z       Forces a new zone transfer using all domains, servers,

                        and associated TSIG keys in DB

  --ADD-DOMAIN DOMAINDETAILS, -zA DOMAINDETAILS

                        Add a domain in the form <Domain name>,<Server>,<TSIG

                        Key Name>.

  --DELETE-DOMAIN DOMAINTODELETE, -zD DOMAINTODELETE

                        Delete a DNS domain from the database by name.

  --IMPORT-ZONE-FILE IMPORTZONEFILE, -zI IMPORTZONEFILE

                        Add domains for zone transfers from a file.

  --CLEAR-DOMAINS, -zC  Clears all DNS domains stored in DB.

  --LIST-DOMAINS, -zL   Lists all DNS domains stored in DB.

  --LIST-OUTSIDE, -e    Prints a list of all names and IPs from our zone

                        transfers that are outside defined net ranges.

  --SQL-CREDS SQLCREDSFILE, -q SQLCREDSFILE

                        Use SQL credentials in file at specified path.

工具使用样例

搜索目标Web技术相关的字符串或正则表达式：

./webstor.py -sP "content=\"wordpress 4.[7-9]"

上述命令将枚举该正则表达式对应的网站列表，期待的响应来自于Wordpress v4.7-9站点。

如果需要将上述正则表达式存储为自定义指纹的话，就可以通过名称来查询了，而无需每次都输入正则表达式：

./webstor.py -c "wordpress4.7-9,content=\"wordpress 4.[7-9]"

上述命令运行之后，我们就可以使用下列命令来让WebStor执行查询了：

./webstor.py -sC wordpress4.7-9

通过Wappalyzer定义来让WebStor查询WordPress站点：

./webstor.py -sW wordpress

此时，WebStor将返回目标组织网络中所有可抵达的WordPress站点信息。注意，Wappalyzer的扫描速度会比使用模式/正则表达式的速度要慢，具体取决于待验证的查询属性数量。

许可证协议

本项目的开发与发布遵循MIT开源许可证协议。

项目地址

WebStor：【GitHub传送门】