mirror of
https://github.com/jumpserver/jumpserver.git
synced 2025-12-15 08:32:48 +00:00
Compare commits
83 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
ce2bbf08e7 | ||
|
|
58c00ca09d | ||
|
|
1e35fee1c9 | ||
|
|
87189412fe | ||
|
|
471cb45535 | ||
|
|
11b0aa3b12 | ||
|
|
4b1b63f7b8 | ||
|
|
512534715b | ||
|
|
761ff5091a | ||
|
|
87894df126 | ||
|
|
d042de7b09 | ||
|
|
5e6e97c822 | ||
|
|
f146873501 | ||
|
|
35dfdf831a | ||
|
|
2b31cb2806 | ||
|
|
e43ffa7994 | ||
|
|
b0a9a83231 | ||
|
|
7da14571ac | ||
|
|
73b67da4c0 | ||
|
|
4bf2371cf0 | ||
|
|
075cbc497b | ||
|
|
1a0d9a20f9 | ||
|
|
fdb8416cac | ||
|
|
e2d5b69510 | ||
|
|
9944474ba0 | ||
|
|
ce6b9de07c | ||
|
|
b97759687d | ||
|
|
68b6236de2 | ||
|
|
6616374c30 | ||
|
|
682f6b2fb9 | ||
|
|
a2e3979916 | ||
|
|
f11d3c1cf2 | ||
|
|
f0bad5f107 | ||
|
|
ad3bc72dfb | ||
|
|
de9c69843d | ||
|
|
d2678e2a43 | ||
|
|
632ea87f07 | ||
|
|
4e7e1d5e15 | ||
|
|
1ac8537a34 | ||
|
|
dcaa798c2e | ||
|
|
8da4027e32 | ||
|
|
32e2d19553 | ||
|
|
48d1eecc08 | ||
|
|
0ab88ce754 | ||
|
|
bee5500425 | ||
|
|
7c03af7668 | ||
|
|
7a61a671a2 | ||
|
|
4a1fc0e2ac | ||
|
|
1e5e87e62a | ||
|
|
96c3b81383 | ||
|
|
297fedeffa | ||
|
|
9cd5675209 | ||
|
|
a5179d1596 | ||
|
|
c2463fe573 | ||
|
|
2f8042141c | ||
|
|
06a4e0d395 | ||
|
|
bb9d92fd7e | ||
|
|
749f9d3f81 | ||
|
|
03ad7777d0 | ||
|
|
7e4f20f443 | ||
|
|
607b7fd29f | ||
|
|
8895763ab4 | ||
|
|
8b1e202e68 | ||
|
|
32fe8f674c | ||
|
|
b4ef7bef55 | ||
|
|
31982c6547 | ||
|
|
67d3b63c6d | ||
|
|
f34fb5d9d5 | ||
|
|
3ec78ff9be | ||
|
|
f361621ab5 | ||
|
|
cd9587f68e | ||
|
|
2ff01a4bb3 | ||
|
|
06ed358fbc | ||
|
|
3e11249e8c | ||
|
|
6b5435b768 | ||
|
|
7d5a13de38 | ||
|
|
07bd44990b | ||
|
|
e4938ffc85 | ||
|
|
85d226eb07 | ||
|
|
c9a9ca7923 | ||
|
|
306f7a08d1 | ||
|
|
b86f9ac871 | ||
|
|
2562386fe0 |
@@ -1,7 +1,7 @@
|
||||
# JumpServer 多云环境下更好用的堡垒机
|
||||
|
||||
[](https://www.python.org/)
|
||||
[](https://www.djangoproject.com/)
|
||||
[](https://www.gnu.org/licenses/old-licenses/gpl-2.0.html)
|
||||
[](https://github.com/jumpserver/jumpserver/releases)
|
||||
[](https://hub.docker.com/u/jumpserver)
|
||||
|
||||
- [ENGLISH](https://github.com/jumpserver/jumpserver/blob/master/README_EN.md)
|
||||
|
||||
390
README_EN.md
390
README_EN.md
@@ -1,143 +1,245 @@
|
||||
## Jumpserver
|
||||
# Jumpserver - The Bastion Host for Multi-Cloud Environment
|
||||
|
||||
[](https://www.python.org/)
|
||||
[](https://www.djangoproject.com/)
|
||||
[](https://hub.docker.com/u/jumpserver)
|
||||
|
||||
----
|
||||
## CRITICAL BUG WARNING
|
||||
- [中文版](https://github.com/jumpserver/jumpserver/blob/master/README.md)
|
||||
|
||||
Recently we have found a critical bug for remote execution vulnerability which leads to pre-auth and info leak, please fix it as soon as possible.
|
||||
|
||||
Thanks for **reactivity from Alibaba Hackerone bug bounty program** report us this bug
|
||||
|
||||
**Vulnerable version:**
|
||||
```
|
||||
< v2.6.2
|
||||
< v2.5.4
|
||||
< v2.4.5
|
||||
= v1.5.9
|
||||
>= v1.5.3
|
||||
```
|
||||
|
||||
**Safe and Stable version:**
|
||||
```
|
||||
>= v2.6.2
|
||||
>= v2.5.4
|
||||
>= v2.4.5
|
||||
= v1.5.9 (version tag didn't change)
|
||||
< v1.5.3
|
||||
```
|
||||
|
||||
**Bug Fix Solution:**
|
||||
Upgrade to the latest version or the version mentioned above
|
||||
|
||||
|
||||
**Temporary Solution (upgrade asap):**
|
||||
|
||||
Modify the Nginx config file and disable the vulnerable api listed below
|
||||
|
||||
```
|
||||
/api/v1/authentication/connection-token/
|
||||
/api/v1/users/connection-token/
|
||||
```
|
||||
|
||||
Path to Nginx config file
|
||||
|
||||
```
|
||||
# Previous Community version
|
||||
/etc/nginx/conf.d/jumpserver.conf
|
||||
|
||||
# Previous Enterprise version
|
||||
jumpserver-release/nginx/http_server.conf
|
||||
|
||||
# Latest version
|
||||
jumpserver-release/compose/config_static/http_server.conf
|
||||
```
|
||||
|
||||
Changes in Nginx config file
|
||||
|
||||
```
|
||||
### Put the following code on top of location server, or before /api and /
|
||||
location /api/v1/authentication/connection-token/ {
|
||||
return 403;
|
||||
}
|
||||
|
||||
location /api/v1/users/connection-token/ {
|
||||
return 403;
|
||||
}
|
||||
### End right here
|
||||
|
||||
location /api/ {
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_pass http://core:8080;
|
||||
}
|
||||
|
||||
...
|
||||
```
|
||||
|
||||
Save the file and restart Nginx
|
||||
|
||||
```
|
||||
docker deployment:
|
||||
$ docker restart jms_nginx
|
||||
|
||||
rpm or other deployment:
|
||||
$ systemctl restart nginx
|
||||
|
||||
```
|
||||
|
||||
**Bug Fix Verification**
|
||||
|
||||
```
|
||||
# Download the following script to check if it is fixed
|
||||
$ wget https://github.com/jumpserver/jumpserver/releases/download/v2.6.2/jms_bug_check.sh
|
||||
|
||||
# Run the code to verify it
|
||||
$ bash jms_bug_check.sh demo.jumpserver.org
|
||||
漏洞已修复 (It means the bug is fixed)
|
||||
漏洞未修复 (It means the bug is not fixed and the system is still vulnerable)
|
||||
```
|
||||
|
||||
|
||||
**Attack Simulation**
|
||||
|
||||
Go to the logs directory which should contain gunicorn.log file. Then download the "attack" script and execute it
|
||||
|
||||
```
|
||||
$ pwd
|
||||
/opt/jumpserver/core/logs
|
||||
|
||||
$ ls gunicorn.log
|
||||
gunicorn.log
|
||||
|
||||
$ wget 'https://github.com/jumpserver/jumpserver/releases/download/v2.6.2/jms_check_attack.sh'
|
||||
$ bash jms_check_attack.sh
|
||||
系统未被入侵 (It means the system is safe)
|
||||
系统已被入侵 (It means the system is being attacked)
|
||||
```
|
||||
|Security Notice|
|
||||
|------------------|
|
||||
|On 15th January 2021, JumpServer found a critical bug for remote execution vulnerability. Please fix it asap! [For more detail](https://github.com/jumpserver/jumpserver/issues/5533) Thanks for **reactivity of Alibaba Hackerone bug bounty program** report use the bug|
|
||||
|
||||
--------------------------
|
||||
|
||||
----
|
||||
|
||||
- [中文版](https://github.com/jumpserver/jumpserver/blob/master/README.md)
|
||||
|
||||
Jumpserver is the world's first open-source PAM (Privileged Access Management System) and is licensed under the GNU GPL v2.0. It is a 4A-compliant professional operation and maintenance security audit system.
|
||||
Jumpserver is the world's first open-source Bastion Host and is licensed under the GNU GPL v2.0. It is a 4A-compliant professional operation and maintenance security audit system.
|
||||
|
||||
Jumpserver uses Python / Django for development, follows Web 2.0 specifications, and is equipped with an industry-leading Web Terminal solution that provides a beautiful user interface and great user experience
|
||||
|
||||
Jumpserver adopts a distributed architecture to support multi-branch deployment across multiple cross-regional areas. The central node provides APIs, and login nodes are deployed in each branch. It can be scaled horizontally without concurrency restrictions.
|
||||
|
||||
Change the world, starting from little things.
|
||||
Change the world by taking every little step
|
||||
|
||||
----
|
||||
### Advantages
|
||||
|
||||
### Features
|
||||
- Open Source: huge transparency and free to access with quick installation process.
|
||||
- Distributed: support large-scale concurrent access with ease.
|
||||
- No Plugin required: all you need is a browser, the ultimate Web Terminal experience.
|
||||
- Multi-Cloud supported: a unified system to manage assets on different clouds at the same time
|
||||
- Cloud storage: audit records are stored in the cloud. Data lost no more!
|
||||
- Multi-Tenant system: multiple subsidiary companies or departments access the same system simultaneously.
|
||||
- Many applications supported: link to databases, windows remote applications, and Kubernetes cluster, etc.
|
||||
|
||||

|
||||
## Features List
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<td rowspan="8">Authentication</td>
|
||||
<td rowspan="5">Login</td>
|
||||
<td>Unified way to access and authenticate resources</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>LDAP/AD Authentication</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RADIUS Authentication</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>OpenID Authentication(Single Sign-On)</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>CAS Authentication (Single Sign-On)</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="2">MFA (Multi-Factor Authentication)</td>
|
||||
<td>Use Google Authenticator for MFA</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RADIUS (Remote Authentication Dial In User Service)</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Login Supervision</td>
|
||||
<td>Any user’s login behavior is supervised and controlled by the administrator:small_orange_diamond:</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="11">Accounting</td>
|
||||
<td rowspan="2">Centralized Accounts Management</td>
|
||||
<td>Admin Users management</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>System Users management</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="4">Unified Password Management</td>
|
||||
<td>Asset password custody (a matrix storing all asset password with dense security)</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Auto-generated passwords</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Automatic password handling (auto login assets)</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Password expiration settings</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="2">Password change Schedular</td>
|
||||
<td>Support regular batch Linux/Windows assets password changing:small_orange_diamond:</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Implement multiple password strategies:small_orange_diamond:</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Multi-Cloud Management</td>
|
||||
<td>Automatically manage private cloud and public cloud assets in a unified platform :small_orange_diamond:</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Users Acquisition </td>
|
||||
<td>Create regular custom tasks to collect system users in selected assets to identify and track the privileges ownership:small_orange_diamond:</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Password Vault </td>
|
||||
<td>Unified operations to check, update, and test system user password to prevent stealing or unauthorised sharing of passwords:small_orange_diamond:</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="15">Authorization</td>
|
||||
<td>Multi-Dimensional</td>
|
||||
<td>Granting users or user groups to access assets, asset nodes, or applications through system users. Providing precise access control to different roles of users</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="4">Assets</td>
|
||||
<td>Assets are arranged and displayed in a tree structure </td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Assets and Nodes have immense flexibility for authorizing</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Assets in nodes inherit authorization automatically</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>child nodes automatically inherit authorization from parent nodes</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="2">Application</td>
|
||||
<td>Provides granular access control for privileged users on application level to protect from unauthorized access and unintentional errors</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Database applications (MySQL, Oracle, PostgreSQL, MariaDB, etc.) and Remote App:small_orange_diamond: </td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Actions</td>
|
||||
<td>Deeper restriction on the control of file upload, download and connection actions of authorized assets. Control the permission of clipboard copy/paste (from outer terminal to current asset)</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Time Bound</td>
|
||||
<td>Sharply limited the available (accessible) time for account access to the authorized resources to reduce the risk and attack surface drastically</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Privileged Assignment</td>
|
||||
<td>Assign the denied/allowed command lists to different system users as privilege elevation, with the latter taking the form of allowing particular commands to be run with a higher level of privileges. (Minimize insider threat)</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Command Filtering</td>
|
||||
<td>Creating list of restriction commands that you would like to assign to different authorized system users for filtering purpose</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>File Transfer and Management</td>
|
||||
<td>Support SFTP file upload/download</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>File Management</td>
|
||||
<td>Provide a Web UI for SFTP file management</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Workflow Management</td>
|
||||
<td>Manage user login confirmation requests and assets or applications authorization requests for Just-In-Time Privileges functionality:small_orange_diamond:</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Group Management </td>
|
||||
<td>Establishing a multi-tenant ecosystem that able authority isolation to keep malicious actors away from sensitive administrative backends:small_orange_diamond:</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="8">Auditing</td>
|
||||
<td>Operations</td>
|
||||
<td>Auditing user operation behaviors for any access or usage of given privileged accounts</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="2">Session</td>
|
||||
<td>Support real-time session audit</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Full history of all previous session audits</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3">Video</td>
|
||||
<td>Complete session audit and playback recordings on assets operation (Linux, Windows)</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Full recordings of RemoteApp, MySQL, and Kubernetes:small_orange_diamond:</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Supports uploading recordings to public clouds</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Command</td>
|
||||
<td>Command auditing on assets and applications operation. Send warning alerts when executing illegal commands</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>File Transfer</td>
|
||||
<td>Full recordings of file upload and download</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="20">Database</td>
|
||||
<td rowspan="2">How to connect</td>
|
||||
<td>Command line</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Built-in Web UI:small_orange_diamond:</td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
<td rowspan="4">Supported Database</td>
|
||||
<td>MySQL</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Oracle :small_orange_diamond:</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MariaDB :small_orange_diamond:</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PostgreSQL :small_orange_diamond:</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="6">Feature Highlights</td>
|
||||
<td>Syntax highlights</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Prettier SQL formmating</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Support Shortcuts</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Support selected SQL statements</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SQL commands history query</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Support page creation: DB, TABLE</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="2">Session Auditing</td>
|
||||
<td>Full records of command</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Playback videos</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
**Note**: Rows with :small_orange_diamond: at the end of the sentence means that it is X-PACK features exclusive ([Apply for X-PACK Trial](https://jinshuju.net/f/kyOYpi))
|
||||
|
||||
### Start
|
||||
|
||||
@@ -162,6 +264,50 @@ We provide the SDK for your other systems to quickly interact with the Jumpserve
|
||||
- [Python](https://github.com/jumpserver/jumpserver-python-sdk) Jumpserver other components use this SDK to complete the interaction.
|
||||
- [Java](https://github.com/KaiJunYan/jumpserver-java-sdk.git) Thanks to 恺珺 for providing his Java SDK vesrion.
|
||||
|
||||
## JumpServer Component Projects
|
||||
- [Lina](https://github.com/jumpserver/lina) JumpServer Web UI
|
||||
- [Luna](https://github.com/jumpserver/luna) JumpServer Web Terminal
|
||||
- [KoKo](https://github.com/jumpserver/koko) JumpServer Character protocaol Connector, replace original Python Version [Coco](https://github.com/jumpserver/coco)
|
||||
- [Guacamole](https://github.com/jumpserver/docker-guacamole) JumpServer Graphics protocol Connector,rely on [Apache Guacamole](https://guacamole.apache.org/)
|
||||
|
||||
## Contribution
|
||||
If you have any good ideas or helping us to fix bugs, please submit a Pull Request and accept our thanks :)
|
||||
|
||||
Thanks to the following contributors for making JumpServer better everyday!
|
||||
|
||||
<a href="https://github.com/jumpserver/jumpserver/graphs/contributors">
|
||||
<img src="https://contrib.rocks/image?repo=jumpserver/jumpserver" />
|
||||
</a>
|
||||
|
||||
|
||||
## Thanks to
|
||||
- [Apache Guacamole](https://guacamole.apache.org/) Web page connection RDP, SSH, VNC protocol equipment. JumpServer graphical connection dependent.
|
||||
- [OmniDB](https://omnidb.org/) Web page connection to databases. JumpServer Web database dependent.
|
||||
|
||||
|
||||
## JumpServer Enterprise Version
|
||||
- [Apply for it](https://jinshuju.net/f/kyOYpi)
|
||||
|
||||
## Case Study
|
||||
|
||||
- [JumpServer 堡垒机护航顺丰科技超大规模资产安全运维](https://blog.fit2cloud.com/?p=1147);
|
||||
- [JumpServer 堡垒机让“大智慧”的混合 IT 运维更智慧](https://blog.fit2cloud.com/?p=882);
|
||||
- [携程 JumpServer 堡垒机部署与运营实战](https://blog.fit2cloud.com/?p=851);
|
||||
- [小红书的JumpServer堡垒机大规模资产跨版本迁移之路](https://blog.fit2cloud.com/?p=516);
|
||||
- [JumpServer堡垒机助力中手游提升多云环境下安全运维能力](https://blog.fit2cloud.com/?p=732);
|
||||
- [中通快递:JumpServer主机安全运维实践](https://blog.fit2cloud.com/?p=708);
|
||||
- [东方明珠:JumpServer高效管控异构化、分布式云端资产](https://blog.fit2cloud.com/?p=687);
|
||||
- [江苏农信:JumpServer堡垒机助力行业云安全运维](https://blog.fit2cloud.com/?p=666)。
|
||||
|
||||
## For safety instructions
|
||||
|
||||
JumpServer is a security product. Please refer to [Basic Security Recommendations](https://docs.jumpserver.org/zh/master/install/install_security/) for deployment and installation.
|
||||
|
||||
If you find a security problem, please contact us directly:
|
||||
|
||||
- ibuler@fit2cloud.com
|
||||
- support@fit2cloud.com
|
||||
- 400-052-0755
|
||||
|
||||
### License & Copyright
|
||||
Copyright (c) 2014-2019 Beijing Duizhan Tech, Inc., All rights reserved.
|
||||
|
||||
@@ -1,9 +0,0 @@
|
||||
from django.utils.translation import ugettext as _
|
||||
|
||||
|
||||
common_help_text = _('Format for comma-delimited string, with * indicating a match all. ')
|
||||
|
||||
ip_group_help_text = common_help_text + _(
|
||||
'Such as: '
|
||||
'192.168.10.1, 192.168.1.0/24, 10.1.1.1-10.1.1.20, 2001:db8:2de::e13, 2001:db8:1a:1110::/64 '
|
||||
)
|
||||
@@ -33,6 +33,9 @@ class LoginACL(BaseACL):
|
||||
class Meta:
|
||||
ordering = ('priority', '-date_updated', 'name')
|
||||
|
||||
def __str__(self):
|
||||
return self.name
|
||||
|
||||
@property
|
||||
def action_reject(self):
|
||||
return self.action == self.ActionChoices.reject
|
||||
|
||||
@@ -38,6 +38,9 @@ class LoginAssetACL(BaseACL, OrgModelMixin):
|
||||
unique_together = ('name', 'org_id')
|
||||
ordering = ('priority', '-date_updated', 'name')
|
||||
|
||||
def __str__(self):
|
||||
return self.name
|
||||
|
||||
@classmethod
|
||||
def filter(cls, user, asset, system_user, action):
|
||||
queryset = cls.objects.filter(action=action)
|
||||
|
||||
@@ -4,7 +4,6 @@ from common.drf.serializers import BulkModelSerializer
|
||||
from orgs.utils import current_org
|
||||
from ..models import LoginACL
|
||||
from ..utils import is_ip_address, is_ip_network, is_ip_segment
|
||||
from .. import const
|
||||
|
||||
|
||||
__all__ = ['LoginACLSerializer', ]
|
||||
@@ -21,8 +20,14 @@ def ip_group_child_validator(ip_group_child):
|
||||
|
||||
|
||||
class LoginACLSerializer(BulkModelSerializer):
|
||||
ip_group_help_text = _(
|
||||
'Format for comma-delimited string, with * indicating a match all. '
|
||||
'Such as: '
|
||||
'192.168.10.1, 192.168.1.0/24, 10.1.1.1-10.1.1.20, 2001:db8:2de::e13, 2001:db8:1a:1110::/64 '
|
||||
)
|
||||
|
||||
ip_group = serializers.ListField(
|
||||
default=['*'], label=_('IP'), help_text=const.ip_group_help_text,
|
||||
default=['*'], label=_('IP'), help_text=ip_group_help_text,
|
||||
child=serializers.CharField(max_length=1024, validators=[ip_group_child_validator])
|
||||
)
|
||||
user_display = serializers.ReadOnlyField(source='user.name', label=_('User'))
|
||||
|
||||
@@ -1,46 +1,60 @@
|
||||
from rest_framework import serializers
|
||||
from django.utils.translation import ugettext as _
|
||||
from django.utils.translation import ugettext_lazy as _
|
||||
from orgs.mixins.serializers import BulkOrgResourceModelSerializer
|
||||
from assets.models import SystemUser
|
||||
from acls import models
|
||||
from orgs.models import Organization
|
||||
from .. import const
|
||||
|
||||
|
||||
__all__ = ['LoginAssetACLSerializer']
|
||||
|
||||
|
||||
common_help_text = _('Format for comma-delimited string, with * indicating a match all. ')
|
||||
|
||||
|
||||
class LoginAssetACLUsersSerializer(serializers.Serializer):
|
||||
username_group = serializers.ListField(
|
||||
default=['*'], child=serializers.CharField(max_length=128), label=_('Username'),
|
||||
help_text=const.common_help_text
|
||||
help_text=common_help_text
|
||||
)
|
||||
|
||||
|
||||
class LoginAssetACLAssestsSerializer(serializers.Serializer):
|
||||
ip_group_help_text = _(
|
||||
'Format for comma-delimited string, with * indicating a match all. '
|
||||
'Such as: '
|
||||
'192.168.10.1, 192.168.1.0/24, 10.1.1.1-10.1.1.20, 2001:db8:2de::e13, 2001:db8:1a:1110::/64 '
|
||||
'(Domain name support)'
|
||||
)
|
||||
|
||||
ip_group = serializers.ListField(
|
||||
default=['*'], child=serializers.CharField(max_length=1024), label=_('IP'),
|
||||
help_text=const.ip_group_help_text + _('(Domain name support)')
|
||||
help_text=ip_group_help_text
|
||||
)
|
||||
hostname_group = serializers.ListField(
|
||||
default=['*'], child=serializers.CharField(max_length=128), label=_('Hostname'),
|
||||
help_text=const.common_help_text
|
||||
help_text=common_help_text
|
||||
)
|
||||
|
||||
|
||||
class LoginAssetACLSystemUsersSerializer(serializers.Serializer):
|
||||
protocol_group_help_text = _(
|
||||
'Format for comma-delimited string, with * indicating a match all. '
|
||||
'Protocol options: {}'
|
||||
)
|
||||
|
||||
name_group = serializers.ListField(
|
||||
default=['*'], child=serializers.CharField(max_length=128), label=_('Name'),
|
||||
help_text=const.common_help_text
|
||||
help_text=common_help_text
|
||||
)
|
||||
username_group = serializers.ListField(
|
||||
default=['*'], child=serializers.CharField(max_length=128), label=_('Username'),
|
||||
help_text=const.common_help_text
|
||||
help_text=common_help_text
|
||||
)
|
||||
protocol_group = serializers.ListField(
|
||||
default=['*'], child=serializers.CharField(max_length=16), label=_('Protocol'),
|
||||
help_text=const.common_help_text + _('Protocol options: {}').format(
|
||||
', '.join(SystemUser.ASSET_CATEGORY_PROTOCOLS)
|
||||
help_text=protocol_group_help_text.format(
|
||||
', '.join([SystemUser.PROTOCOL_SSH, SystemUser.PROTOCOL_TELNET])
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@@ -44,8 +44,8 @@ class ApplicationSerializerMixin(serializers.Serializer):
|
||||
|
||||
|
||||
class ApplicationSerializer(ApplicationSerializerMixin, BulkOrgResourceModelSerializer):
|
||||
category_display = serializers.ReadOnlyField(source='get_category_display', label=_('Category'))
|
||||
type_display = serializers.ReadOnlyField(source='get_type_display', label=_('Type'))
|
||||
category_display = serializers.ReadOnlyField(source='get_category_display', label=_('Category(Display)'))
|
||||
type_display = serializers.ReadOnlyField(source='get_type_display', label=_('Type(Dispaly)'))
|
||||
|
||||
class Meta:
|
||||
model = models.Application
|
||||
|
||||
@@ -28,6 +28,7 @@ from ..tasks import (
|
||||
)
|
||||
from .. import serializers
|
||||
from .mixin import SerializeToTreeNodeMixin
|
||||
from assets.locks import NodeAddChildrenLock
|
||||
|
||||
|
||||
logger = get_logger(__file__)
|
||||
@@ -114,15 +115,16 @@ class NodeChildrenApi(generics.ListCreateAPIView):
|
||||
return super().initial(request, *args, **kwargs)
|
||||
|
||||
def perform_create(self, serializer):
|
||||
data = serializer.validated_data
|
||||
_id = data.get("id")
|
||||
value = data.get("value")
|
||||
if not value:
|
||||
value = self.instance.get_next_child_preset_name()
|
||||
node = self.instance.create_child(value=value, _id=_id)
|
||||
# 避免查询 full value
|
||||
node._full_value = node.value
|
||||
serializer.instance = node
|
||||
with NodeAddChildrenLock(self.instance):
|
||||
data = serializer.validated_data
|
||||
_id = data.get("id")
|
||||
value = data.get("value")
|
||||
if not value:
|
||||
value = self.instance.get_next_child_preset_name()
|
||||
node = self.instance.create_child(value=value, _id=_id)
|
||||
# 避免查询 full value
|
||||
node._full_value = node.value
|
||||
serializer.instance = node
|
||||
|
||||
def get_object(self):
|
||||
pk = self.kwargs.get('pk') or self.request.query_params.get('id')
|
||||
@@ -221,7 +223,8 @@ class NodeAddChildrenApi(generics.UpdateAPIView):
|
||||
serializer_class = serializers.NodeAddChildrenSerializer
|
||||
instance = None
|
||||
|
||||
def put(self, request, *args, **kwargs):
|
||||
def update(self, request, *args, **kwargs):
|
||||
""" 同时支持 put 和 patch 方法"""
|
||||
instance = self.get_object()
|
||||
node_ids = request.data.get("nodes")
|
||||
children = Node.objects.filter(id__in=node_ids)
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
from orgs.utils import current_org
|
||||
from common.utils.lock import DistributedLock
|
||||
from assets.models import Node
|
||||
|
||||
|
||||
class NodeTreeUpdateLock(DistributedLock):
|
||||
@@ -18,3 +19,11 @@ class NodeTreeUpdateLock(DistributedLock):
|
||||
def __init__(self):
|
||||
name = self.get_name()
|
||||
super().__init__(name=name, release_on_transaction_commit=True, reentrant=True)
|
||||
|
||||
|
||||
class NodeAddChildrenLock(DistributedLock):
|
||||
name_template = 'assets.node.add_children.<org_id:{org_id}>'
|
||||
|
||||
def __init__(self, node: Node):
|
||||
name = self.name_template.format(org_id=node.org_id)
|
||||
super().__init__(name=name, release_on_transaction_commit=True)
|
||||
|
||||
61
apps/assets/migrations/0069_change_node_key0_to_key1.py
Normal file
61
apps/assets/migrations/0069_change_node_key0_to_key1.py
Normal file
@@ -0,0 +1,61 @@
|
||||
from django.db import migrations
|
||||
from django.db.transaction import atomic
|
||||
|
||||
default_id = '00000000-0000-0000-0000-000000000002'
|
||||
|
||||
|
||||
def change_key0_to_key1(apps, schema_editor):
|
||||
from orgs.utils import set_current_org
|
||||
|
||||
# https://stackoverflow.com/questions/28777338/django-migrations-runpython-not-able-to-call-model-methods
|
||||
Organization = apps.get_model('orgs', 'Organization')
|
||||
Node = apps.get_model('assets', 'Node')
|
||||
|
||||
print()
|
||||
org = Organization.objects.get(id=default_id)
|
||||
set_current_org(org)
|
||||
|
||||
exists_0 = Node.objects.filter(key__startswith='0').exists()
|
||||
if not exists_0:
|
||||
print(f'--> Not exist key=0 nodes, do nothing.')
|
||||
return
|
||||
|
||||
key_1_count = Node.objects.filter(key__startswith='1').count()
|
||||
if key_1_count > 1:
|
||||
print(f'--> Node key=1 have children, can`t just delete it. Please contact JumpServer team')
|
||||
return
|
||||
|
||||
root_node = Node.objects.filter(key='1').first()
|
||||
if root_node and root_node.assets.exists():
|
||||
print(f'--> Node key=1 has assets, do nothing.')
|
||||
return
|
||||
|
||||
with atomic():
|
||||
if root_node:
|
||||
print(f'--> Delete node key=1')
|
||||
root_node.delete()
|
||||
|
||||
nodes_0 = Node.objects.filter(key__startswith='0')
|
||||
|
||||
for n in nodes_0:
|
||||
old_key = n.key
|
||||
key_list = n.key.split(':')
|
||||
key_list[0] = '1'
|
||||
new_key = ':'.join(key_list)
|
||||
new_parent_key = ':'.join(key_list[:-1])
|
||||
n.key = new_key
|
||||
n.parent_key = new_parent_key
|
||||
n.save()
|
||||
print('--> Modify key ( {} > {} )'.format(old_key, new_key))
|
||||
|
||||
|
||||
class Migration(migrations.Migration):
|
||||
|
||||
dependencies = [
|
||||
('orgs', '0010_auto_20210219_1241'),
|
||||
('assets', '0068_auto_20210312_1455'),
|
||||
]
|
||||
|
||||
operations = [
|
||||
migrations.RunPython(change_key0_to_key1)
|
||||
]
|
||||
@@ -113,7 +113,7 @@ class AuthMixin:
|
||||
if self.public_key:
|
||||
public_key = self.public_key
|
||||
elif self.private_key:
|
||||
public_key = ssh_pubkey_gen(self.private_key, self.password)
|
||||
public_key = ssh_pubkey_gen(private_key=self.private_key, password=self.password)
|
||||
else:
|
||||
return ''
|
||||
|
||||
|
||||
@@ -307,6 +307,15 @@ class NodeAllAssetsMappingMixin:
|
||||
org_id = str(org_id)
|
||||
cls.orgid_nodekey_assetsid_mapping.pop(org_id, None)
|
||||
|
||||
@classmethod
|
||||
def expire_all_orgs_node_all_asset_ids_mapping_from_memory(cls):
|
||||
orgs = Organization.objects.all()
|
||||
org_ids = [str(org.id) for org in orgs]
|
||||
org_ids.append(Organization.ROOT_ID)
|
||||
|
||||
for id in org_ids:
|
||||
cls.expire_node_all_asset_ids_mapping_from_memory(id)
|
||||
|
||||
# get order: from memory -> (from cache -> to generate)
|
||||
@classmethod
|
||||
def get_node_all_asset_ids_mapping_from_cache_or_generate_to_cache(cls, org_id):
|
||||
|
||||
@@ -13,6 +13,7 @@ from common.signals import django_ready
|
||||
from common.utils.connection import RedisPubSub
|
||||
from common.utils import get_logger
|
||||
from assets.models import Asset, Node
|
||||
from orgs.models import Organization
|
||||
|
||||
|
||||
logger = get_logger(__file__)
|
||||
@@ -36,13 +37,18 @@ node_assets_mapping_for_memory_pub_sub = NodeAssetsMappingForMemoryPubSub()
|
||||
def expire_node_assets_mapping_for_memory(org_id):
|
||||
# 所有进程清除(自己的 memory 数据)
|
||||
org_id = str(org_id)
|
||||
node_assets_mapping_for_memory_pub_sub.publish(org_id)
|
||||
root_org_id = Organization.ROOT_ID
|
||||
|
||||
# 当前进程清除(cache 数据)
|
||||
logger.debug(
|
||||
"Expire node assets id mapping from cache of org={}, pid={}"
|
||||
"".format(org_id, os.getpid())
|
||||
)
|
||||
Node.expire_node_all_asset_ids_mapping_from_cache(org_id)
|
||||
Node.expire_node_all_asset_ids_mapping_from_cache(root_org_id)
|
||||
|
||||
node_assets_mapping_for_memory_pub_sub.publish(org_id)
|
||||
node_assets_mapping_for_memory_pub_sub.publish(root_org_id)
|
||||
|
||||
|
||||
@receiver(post_save, sender=Node)
|
||||
@@ -73,16 +79,22 @@ def subscribe_node_assets_mapping_expire(sender, **kwargs):
|
||||
logger.debug("Start subscribe for expire node assets id mapping from memory")
|
||||
|
||||
def keep_subscribe():
|
||||
subscribe = node_assets_mapping_for_memory_pub_sub.subscribe()
|
||||
for message in subscribe.listen():
|
||||
if message["type"] != "message":
|
||||
continue
|
||||
org_id = message['data'].decode()
|
||||
Node.expire_node_all_asset_ids_mapping_from_memory(org_id)
|
||||
logger.debug(
|
||||
"Expire node assets id mapping from memory of org={}, pid={}"
|
||||
"".format(str(org_id), os.getpid())
|
||||
)
|
||||
while True:
|
||||
try:
|
||||
subscribe = node_assets_mapping_for_memory_pub_sub.subscribe()
|
||||
for message in subscribe.listen():
|
||||
if message["type"] != "message":
|
||||
continue
|
||||
org_id = message['data'].decode()
|
||||
Node.expire_node_all_asset_ids_mapping_from_memory(org_id)
|
||||
logger.debug(
|
||||
"Expire node assets id mapping from memory of org={}, pid={}"
|
||||
"".format(str(org_id), os.getpid())
|
||||
)
|
||||
except Exception as e:
|
||||
logger.exception(f'subscribe_node_assets_mapping_expire: {e}')
|
||||
Node.expire_all_orgs_node_all_asset_ids_mapping_from_memory()
|
||||
|
||||
t = threading.Thread(target=keep_subscribe)
|
||||
t.daemon = True
|
||||
t.start()
|
||||
|
||||
@@ -56,6 +56,7 @@ def get_push_unixlike_system_user_tasks(system_user, username=None):
|
||||
'shell': system_user.shell or Empty,
|
||||
'state': 'present',
|
||||
'home': system_user.home or Empty,
|
||||
'expires': -1,
|
||||
'groups': groups or Empty,
|
||||
'comment': comment
|
||||
}
|
||||
|
||||
18
apps/audits/migrations/0012_auto_20210414_1443.py
Normal file
18
apps/audits/migrations/0012_auto_20210414_1443.py
Normal file
@@ -0,0 +1,18 @@
|
||||
# Generated by Django 3.1 on 2021-04-14 06:43
|
||||
|
||||
from django.db import migrations, models
|
||||
|
||||
|
||||
class Migration(migrations.Migration):
|
||||
|
||||
dependencies = [
|
||||
('audits', '0011_userloginlog_backend'),
|
||||
]
|
||||
|
||||
operations = [
|
||||
migrations.AlterField(
|
||||
model_name='userloginlog',
|
||||
name='type',
|
||||
field=models.CharField(choices=[('W', 'Web'), ('T', 'Terminal'), ('U', 'Unknown')], max_length=2, verbose_name='Login type'),
|
||||
),
|
||||
]
|
||||
@@ -79,6 +79,7 @@ class UserLoginLog(models.Model):
|
||||
LOGIN_TYPE_CHOICE = (
|
||||
('W', 'Web'),
|
||||
('T', 'Terminal'),
|
||||
('U', 'Unknown'),
|
||||
)
|
||||
|
||||
MFA_DISABLED = 0
|
||||
|
||||
@@ -27,11 +27,23 @@ json_render = JSONRenderer()
|
||||
|
||||
|
||||
MODELS_NEED_RECORD = (
|
||||
'User', 'UserGroup', 'Asset', 'Node', 'AdminUser', 'SystemUser',
|
||||
'Domain', 'Gateway', 'Organization', 'AssetPermission', 'CommandFilter',
|
||||
'CommandFilterRule', 'License', 'Setting', 'Account', 'SyncInstanceTask',
|
||||
'Platform', 'ChangeAuthPlan', 'GatherUserTask',
|
||||
'RemoteApp', 'RemoteAppPermission', 'DatabaseApp', 'DatabaseAppPermission',
|
||||
# users
|
||||
'User', 'UserGroup',
|
||||
# acls
|
||||
'LoginACL', 'LoginAssetACL',
|
||||
# assets
|
||||
'Asset', 'Node', 'AdminUser', 'SystemUser', 'Domain', 'Gateway', 'CommandFilterRule',
|
||||
'CommandFilter', 'Platform',
|
||||
# applications
|
||||
'Application',
|
||||
# orgs
|
||||
'Organization',
|
||||
# settings
|
||||
'Setting',
|
||||
# perms
|
||||
'AssetPermission', 'ApplicationPermission',
|
||||
# xpack
|
||||
'License', 'Account', 'SyncInstanceTask', 'ChangeAuthPlan', 'GatherUserTask',
|
||||
)
|
||||
|
||||
|
||||
@@ -44,6 +56,7 @@ class AuthBackendLabelMapping(LazyObject):
|
||||
backend_label_mapping[backend] = source.label
|
||||
backend_label_mapping[settings.AUTH_BACKEND_PUBKEY] = _('SSH Key')
|
||||
backend_label_mapping[settings.AUTH_BACKEND_MODEL] = _('Password')
|
||||
backend_label_mapping[settings.AUTH_BACKEND_SSO] = _('SSO')
|
||||
return backend_label_mapping
|
||||
|
||||
def _setup(self):
|
||||
@@ -145,7 +158,7 @@ def generate_data(username, request):
|
||||
user_agent = request.META.get('HTTP_USER_AGENT', '')
|
||||
login_ip = get_request_ip(request) or '0.0.0.0'
|
||||
if isinstance(request, Request):
|
||||
login_type = request.META.get('HTTP_X_JMS_LOGIN_TYPE', '')
|
||||
login_type = request.META.get('HTTP_X_JMS_LOGIN_TYPE', 'U')
|
||||
else:
|
||||
login_type = 'W'
|
||||
|
||||
@@ -153,7 +166,7 @@ def generate_data(username, request):
|
||||
'username': username,
|
||||
'ip': login_ip,
|
||||
'type': login_type,
|
||||
'user_agent': user_agent,
|
||||
'user_agent': user_agent[0:254],
|
||||
'datetime': timezone.now(),
|
||||
'backend': get_login_backend(request)
|
||||
}
|
||||
|
||||
@@ -29,7 +29,7 @@ class MFAChallengeApi(AuthMixin, CreateAPIView):
|
||||
if not valid:
|
||||
self.request.session['auth_mfa'] = ''
|
||||
raise errors.MFAFailedError(
|
||||
username=user.username, request=self.request
|
||||
username=user.username, request=self.request, ip=self.get_request_ip()
|
||||
)
|
||||
else:
|
||||
self.request.session['auth_mfa'] = '1'
|
||||
|
||||
@@ -6,9 +6,7 @@ from django.conf import settings
|
||||
|
||||
from common.exceptions import JMSException
|
||||
from .signals import post_auth_failed
|
||||
from users.utils import (
|
||||
increase_login_failed_count, get_login_failed_count
|
||||
)
|
||||
from users.utils import LoginBlockUtil, MFABlockUtils
|
||||
|
||||
reason_password_failed = 'password_failed'
|
||||
reason_password_decrypt_failed = 'password_decrypt_failed'
|
||||
@@ -18,6 +16,7 @@ reason_user_not_exist = 'user_not_exist'
|
||||
reason_password_expired = 'password_expired'
|
||||
reason_user_invalid = 'user_invalid'
|
||||
reason_user_inactive = 'user_inactive'
|
||||
reason_user_expired = 'user_expired'
|
||||
reason_backend_not_match = 'backend_not_match'
|
||||
reason_acl_not_allow = 'acl_not_allow'
|
||||
|
||||
@@ -30,8 +29,9 @@ reason_choices = {
|
||||
reason_password_expired: _("Password expired"),
|
||||
reason_user_invalid: _('Disabled or expired'),
|
||||
reason_user_inactive: _("This account is inactive."),
|
||||
reason_user_expired: _("This account is expired"),
|
||||
reason_backend_not_match: _("Auth backend not match"),
|
||||
reason_acl_not_allow: _("ACL is not allowed")
|
||||
reason_acl_not_allow: _("ACL is not allowed"),
|
||||
}
|
||||
old_reason_choices = {
|
||||
'0': '-',
|
||||
@@ -52,7 +52,15 @@ block_login_msg = _(
|
||||
"The account has been locked "
|
||||
"(please contact admin to unlock it or try again after {} minutes)"
|
||||
)
|
||||
mfa_failed_msg = _("MFA code invalid, or ntp sync server time")
|
||||
block_mfa_msg = _(
|
||||
"The account has been locked "
|
||||
"(please contact admin to unlock it or try again after {} minutes)"
|
||||
)
|
||||
mfa_failed_msg = _(
|
||||
"MFA code invalid, or ntp sync server time, "
|
||||
"You can also try {times_try} times "
|
||||
"(The account will be temporarily locked for {block_time} minutes)"
|
||||
)
|
||||
|
||||
mfa_required_msg = _("MFA required")
|
||||
mfa_unset_msg = _("MFA not set, please set it first")
|
||||
@@ -80,7 +88,7 @@ class AuthFailedNeedBlockMixin:
|
||||
|
||||
def __init__(self, *args, **kwargs):
|
||||
super().__init__(*args, **kwargs)
|
||||
increase_login_failed_count(self.username, self.ip)
|
||||
LoginBlockUtil(self.username, self.ip).incr_failed_count()
|
||||
|
||||
|
||||
class AuthFailedError(Exception):
|
||||
@@ -107,13 +115,12 @@ class AuthFailedError(Exception):
|
||||
class CredentialError(AuthFailedNeedLogMixin, AuthFailedNeedBlockMixin, AuthFailedError):
|
||||
def __init__(self, error, username, ip, request):
|
||||
super().__init__(error=error, username=username, ip=ip, request=request)
|
||||
times_up = settings.SECURITY_LOGIN_LIMIT_COUNT
|
||||
times_failed = get_login_failed_count(username, ip)
|
||||
times_try = int(times_up) - int(times_failed)
|
||||
util = LoginBlockUtil(username, ip)
|
||||
times_remainder = util.get_remainder_times()
|
||||
block_time = settings.SECURITY_LOGIN_LIMIT_TIME
|
||||
|
||||
default_msg = invalid_login_msg.format(
|
||||
times_try=times_try, block_time=block_time
|
||||
times_try=times_remainder, block_time=block_time
|
||||
)
|
||||
if error == reason_password_failed:
|
||||
self.msg = default_msg
|
||||
@@ -123,12 +130,32 @@ class CredentialError(AuthFailedNeedLogMixin, AuthFailedNeedBlockMixin, AuthFail
|
||||
|
||||
class MFAFailedError(AuthFailedNeedLogMixin, AuthFailedError):
|
||||
error = reason_mfa_failed
|
||||
msg = mfa_failed_msg
|
||||
msg: str
|
||||
|
||||
def __init__(self, username, request):
|
||||
def __init__(self, username, request, ip):
|
||||
util = MFABlockUtils(username, ip)
|
||||
util.incr_failed_count()
|
||||
|
||||
times_remainder = util.get_remainder_times()
|
||||
block_time = settings.SECURITY_LOGIN_LIMIT_TIME
|
||||
|
||||
if times_remainder:
|
||||
self.msg = mfa_failed_msg.format(
|
||||
times_try=times_remainder, block_time=block_time
|
||||
)
|
||||
else:
|
||||
self.msg = block_mfa_msg.format(settings.SECURITY_LOGIN_LIMIT_TIME)
|
||||
super().__init__(username=username, request=request)
|
||||
|
||||
|
||||
class BlockMFAError(AuthFailedNeedLogMixin, AuthFailedError):
|
||||
error = 'block_mfa'
|
||||
|
||||
def __init__(self, username, request, ip):
|
||||
self.msg = block_mfa_msg.format(settings.SECURITY_LOGIN_LIMIT_TIME)
|
||||
super().__init__(username=username, request=request, ip=ip)
|
||||
|
||||
|
||||
class MFAUnsetError(AuthFailedNeedLogMixin, AuthFailedError):
|
||||
error = reason_mfa_unset
|
||||
msg = mfa_unset_msg
|
||||
@@ -184,6 +211,28 @@ class MFARequiredError(NeedMoreInfoError):
|
||||
}
|
||||
|
||||
|
||||
class ACLError(AuthFailedNeedLogMixin, AuthFailedError):
|
||||
msg = reason_acl_not_allow
|
||||
error = 'acl_error'
|
||||
|
||||
def __init__(self, msg, **kwargs):
|
||||
self.msg = msg
|
||||
super().__init__(**kwargs)
|
||||
|
||||
def as_data(self):
|
||||
return {
|
||||
"error": reason_acl_not_allow,
|
||||
"msg": self.msg
|
||||
}
|
||||
|
||||
|
||||
class LoginIPNotAllowed(ACLError):
|
||||
def __init__(self, username, request, **kwargs):
|
||||
self.username = username
|
||||
self.request = request
|
||||
super().__init__(_("IP is not allowed"), **kwargs)
|
||||
|
||||
|
||||
class LoginConfirmBaseError(NeedMoreInfoError):
|
||||
def __init__(self, ticket_id, **kwargs):
|
||||
self.ticket_id = ticket_id
|
||||
|
||||
@@ -15,9 +15,7 @@ from django.shortcuts import reverse
|
||||
|
||||
from common.utils import get_object_or_none, get_request_ip, get_logger, bulk_get
|
||||
from users.models import User
|
||||
from users.utils import (
|
||||
is_block_login, clean_failed_count
|
||||
)
|
||||
from users.utils import LoginBlockUtil, MFABlockUtils
|
||||
from . import errors
|
||||
from .utils import rsa_decrypt
|
||||
from .signals import post_auth_success, post_auth_failed
|
||||
@@ -117,7 +115,7 @@ class AuthMixin:
|
||||
else:
|
||||
username = self.request.POST.get("username")
|
||||
ip = self.get_request_ip()
|
||||
if is_block_login(username, ip):
|
||||
if LoginBlockUtil(username, ip).is_block():
|
||||
logger.warn('Ip was blocked' + ': ' + username + ':' + ip)
|
||||
exception = errors.BlockLoginError(username=username, ip=ip)
|
||||
if raise_exception:
|
||||
@@ -173,7 +171,7 @@ class AuthMixin:
|
||||
if not user:
|
||||
self.raise_credential_error(errors.reason_password_failed)
|
||||
elif user.is_expired:
|
||||
self.raise_credential_error(errors.reason_user_inactive)
|
||||
self.raise_credential_error(errors.reason_user_expired)
|
||||
elif not user.is_active:
|
||||
self.raise_credential_error(errors.reason_user_inactive)
|
||||
return user
|
||||
@@ -183,7 +181,7 @@ class AuthMixin:
|
||||
from acls.models import LoginACL
|
||||
is_allowed = LoginACL.allow_user_to_login(user, ip)
|
||||
if not is_allowed:
|
||||
raise self.raise_credential_error(error=errors.reason_acl_not_allow)
|
||||
raise errors.LoginIPNotAllowed(username=user.username, request=self.request)
|
||||
|
||||
def check_user_auth(self, decrypt_passwd=False):
|
||||
self.check_is_block()
|
||||
@@ -197,7 +195,7 @@ class AuthMixin:
|
||||
self._check_password_require_reset_or_not(user)
|
||||
self._check_passwd_is_too_simple(user, password)
|
||||
|
||||
clean_failed_count(username, ip)
|
||||
LoginBlockUtil(username, ip).clean_failed_count()
|
||||
request.session['auth_password'] = 1
|
||||
request.session['user_id'] = str(user.id)
|
||||
request.session['auto_login'] = auto_login
|
||||
@@ -253,15 +251,34 @@ class AuthMixin:
|
||||
raise errors.MFAUnsetError(user, self.request, url)
|
||||
raise errors.MFARequiredError()
|
||||
|
||||
def mark_mfa_ok(self):
|
||||
self.request.session['auth_mfa'] = 1
|
||||
self.request.session['auth_mfa_time'] = time.time()
|
||||
self.request.session['auth_mfa_type'] = 'otp'
|
||||
|
||||
def check_mfa_is_block(self, username, ip, raise_exception=True):
|
||||
if MFABlockUtils(username, ip).is_block():
|
||||
logger.warn('Ip was blocked' + ': ' + username + ':' + ip)
|
||||
exception = errors.BlockMFAError(username=username, request=self.request, ip=ip)
|
||||
if raise_exception:
|
||||
raise exception
|
||||
else:
|
||||
return exception
|
||||
|
||||
def check_user_mfa(self, code):
|
||||
user = self.get_user_from_session()
|
||||
ip = self.get_request_ip()
|
||||
self.check_mfa_is_block(user.username, ip)
|
||||
ok = user.check_mfa(code)
|
||||
if ok:
|
||||
self.request.session['auth_mfa'] = 1
|
||||
self.request.session['auth_mfa_time'] = time.time()
|
||||
self.request.session['auth_mfa_type'] = 'otp'
|
||||
self.mark_mfa_ok()
|
||||
return
|
||||
raise errors.MFAFailedError(username=user.username, request=self.request)
|
||||
|
||||
raise errors.MFAFailedError(
|
||||
username=user.username,
|
||||
request=self.request,
|
||||
ip=ip
|
||||
)
|
||||
|
||||
def get_ticket(self):
|
||||
from tickets.models import Ticket
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
#
|
||||
from django.utils import timezone
|
||||
from rest_framework import serializers
|
||||
|
||||
from common.utils import get_object_or_none
|
||||
@@ -44,6 +45,10 @@ class BearerTokenSerializer(serializers.Serializer):
|
||||
def get_keyword(obj):
|
||||
return 'Bearer'
|
||||
|
||||
def update_last_login(self, user):
|
||||
user.last_login = timezone.now()
|
||||
user.save(update_fields=['last_login'])
|
||||
|
||||
def create(self, validated_data):
|
||||
request = self.context.get('request')
|
||||
if request.user and not request.user.is_anonymous:
|
||||
@@ -56,6 +61,8 @@ class BearerTokenSerializer(serializers.Serializer):
|
||||
"user id {} not exist".format(user_id)
|
||||
)
|
||||
token, date_expired = user.create_bearer_token(request)
|
||||
self.update_last_login(user)
|
||||
|
||||
instance = {
|
||||
"token": token,
|
||||
"date_expired": date_expired,
|
||||
|
||||
@@ -23,4 +23,3 @@ urlpatterns = [
|
||||
]
|
||||
|
||||
urlpatterns += router.urls
|
||||
|
||||
|
||||
@@ -55,6 +55,9 @@ class UserLoginView(mixins.AuthMixin, FormView):
|
||||
def form_valid(self, form):
|
||||
if not self.request.session.test_cookie_worked():
|
||||
return HttpResponse(_("Please enable cookies and try again."))
|
||||
# https://docs.djangoproject.com/en/3.1/topics/http/sessions/#setting-test-cookies
|
||||
self.request.session.delete_test_cookie()
|
||||
|
||||
try:
|
||||
self.check_user_auth(decrypt_passwd=True)
|
||||
except errors.AuthFailedError as e:
|
||||
@@ -66,6 +69,7 @@ class UserLoginView(mixins.AuthMixin, FormView):
|
||||
new_form = form_cls(data=form.data)
|
||||
new_form._errors = form.errors
|
||||
context = self.get_context_data(form=new_form)
|
||||
self.request.session.set_test_cookie()
|
||||
return self.render_to_response(context)
|
||||
except (errors.PasswdTooSimple, errors.PasswordRequireResetError) as e:
|
||||
return redirect(e.url)
|
||||
|
||||
@@ -22,10 +22,12 @@ class UserLoginOtpView(mixins.AuthMixin, FormView):
|
||||
try:
|
||||
self.check_user_mfa(otp_code)
|
||||
return redirect_to_guard_view()
|
||||
except errors.MFAFailedError as e:
|
||||
except (errors.MFAFailedError, errors.BlockMFAError) as e:
|
||||
form.add_error('otp_code', e.msg)
|
||||
return super().form_invalid(form)
|
||||
except Exception as e:
|
||||
logger.error(e)
|
||||
import traceback
|
||||
traceback.print_exception()
|
||||
return redirect_to_guard_view()
|
||||
|
||||
|
||||
@@ -143,5 +143,5 @@ class BaseFileParser(BaseParser):
|
||||
return data
|
||||
except Exception as e:
|
||||
logger.error(e, exc_info=True)
|
||||
raise ParseError('Parse error! ({})'.format(self.media_type))
|
||||
raise ParseError(_('Parse file error: {}').format(e))
|
||||
|
||||
|
||||
19
apps/common/management/commands/expire_caches.py
Normal file
19
apps/common/management/commands/expire_caches.py
Normal file
@@ -0,0 +1,19 @@
|
||||
from django.core.management.base import BaseCommand
|
||||
|
||||
from assets.signals_handler.node_assets_mapping import expire_node_assets_mapping_for_memory
|
||||
from orgs.models import Organization
|
||||
|
||||
|
||||
def expire_node_assets_mapping():
|
||||
org_ids = Organization.objects.all().values_list('id', flat=True)
|
||||
org_ids = [*org_ids, '00000000-0000-0000-0000-000000000000']
|
||||
|
||||
for org_id in org_ids:
|
||||
expire_node_assets_mapping_for_memory(org_id)
|
||||
|
||||
|
||||
class Command(BaseCommand):
|
||||
help = 'Expire caches'
|
||||
|
||||
def handle(self, *args, **options):
|
||||
expire_node_assets_mapping()
|
||||
@@ -9,6 +9,7 @@ from itertools import chain
|
||||
from django.db.models.signals import m2m_changed
|
||||
from django.core.cache import cache
|
||||
from django.http import JsonResponse
|
||||
from django.utils.translation import ugettext as _
|
||||
from rest_framework.response import Response
|
||||
from rest_framework.settings import api_settings
|
||||
from rest_framework.decorators import action
|
||||
@@ -47,6 +48,9 @@ class RenderToJsonMixin:
|
||||
column_title_field_pairs = jms_context.get('column_title_field_pairs', ())
|
||||
data['title'] = column_title_field_pairs
|
||||
|
||||
if isinstance(request.data, (list, tuple)) and not any(request.data):
|
||||
error = _("Request file format may be wrong")
|
||||
return Response(data={"error": error}, status=400)
|
||||
return Response(data=data)
|
||||
|
||||
|
||||
|
||||
@@ -1,12 +1,14 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
#
|
||||
# coding: utf-8
|
||||
|
||||
from django.contrib.auth.mixins import UserPassesTestMixin
|
||||
from django.utils import timezone
|
||||
|
||||
|
||||
__all__ = ["DatetimeSearchMixin"]
|
||||
|
||||
from rest_framework import permissions
|
||||
|
||||
|
||||
class DatetimeSearchMixin:
|
||||
date_format = '%Y-%m-%d'
|
||||
@@ -36,3 +38,17 @@ class DatetimeSearchMixin:
|
||||
def get(self, request, *args, **kwargs):
|
||||
self.get_date_range()
|
||||
return super().get(request, *args, **kwargs)
|
||||
|
||||
|
||||
class PermissionsMixin(UserPassesTestMixin):
|
||||
permission_classes = [permissions.IsAuthenticated]
|
||||
|
||||
def get_permissions(self):
|
||||
return self.permission_classes
|
||||
|
||||
def test_func(self):
|
||||
permission_classes = self.get_permissions()
|
||||
for permission_class in permission_classes:
|
||||
if not permission_class().has_permission(self.request, self):
|
||||
return False
|
||||
return True
|
||||
@@ -2,7 +2,6 @@
|
||||
#
|
||||
import time
|
||||
from rest_framework import permissions
|
||||
from django.contrib.auth.mixins import UserPassesTestMixin
|
||||
from django.conf import settings
|
||||
|
||||
from orgs.utils import current_org
|
||||
@@ -95,20 +94,6 @@ class WithBootstrapToken(permissions.BasePermission):
|
||||
return settings.BOOTSTRAP_TOKEN == request_bootstrap_token
|
||||
|
||||
|
||||
class PermissionsMixin(UserPassesTestMixin):
|
||||
permission_classes = [permissions.IsAuthenticated]
|
||||
|
||||
def get_permissions(self):
|
||||
return self.permission_classes
|
||||
|
||||
def test_func(self):
|
||||
permission_classes = self.get_permissions()
|
||||
for permission_class in permission_classes:
|
||||
if not permission_class().has_permission(self.request, self):
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
class UserCanAnyPermCurrentOrg(permissions.BasePermission):
|
||||
def has_permission(self, request, view):
|
||||
return current_org.can_any_by(request.user)
|
||||
|
||||
@@ -1 +1,2 @@
|
||||
|
||||
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import time
|
||||
|
||||
from django.core.cache import cache
|
||||
from django.utils import timezone
|
||||
from django.utils.timesince import timesince
|
||||
@@ -6,6 +8,8 @@ from django.http.response import JsonResponse, HttpResponse
|
||||
from rest_framework.views import APIView
|
||||
from rest_framework.permissions import AllowAny
|
||||
from collections import Counter
|
||||
from django.conf import settings
|
||||
from rest_framework.response import Response
|
||||
|
||||
from users.models import User
|
||||
from assets.models import Asset
|
||||
@@ -307,7 +311,68 @@ class IndexApi(TotalCountMixin, DatesLoginMetricMixin, APIView):
|
||||
return JsonResponse(data, status=200)
|
||||
|
||||
|
||||
class PrometheusMetricsApi(APIView):
|
||||
class HealthApiMixin(APIView):
|
||||
def is_token_right(self):
|
||||
token = self.request.query_params.get('token')
|
||||
ok_token = settings.HEALTH_CHECK_TOKEN
|
||||
if ok_token and token != ok_token:
|
||||
return False
|
||||
return True
|
||||
|
||||
def check_permissions(self, request):
|
||||
if not self.is_token_right():
|
||||
msg = 'Health check token error, ' \
|
||||
'Please set query param in url and same with setting HEALTH_CHECK_TOKEN. ' \
|
||||
'eg: $PATH/?token=$HEALTH_CHECK_TOKEN'
|
||||
self.permission_denied(request, message={'error': msg}, code=403)
|
||||
|
||||
|
||||
class HealthCheckView(HealthApiMixin):
|
||||
permission_classes = (AllowAny,)
|
||||
|
||||
@staticmethod
|
||||
def get_db_status():
|
||||
t1 = time.time()
|
||||
try:
|
||||
User.objects.first()
|
||||
t2 = time.time()
|
||||
return True, t2 - t1
|
||||
except:
|
||||
t2 = time.time()
|
||||
return False, t2 - t1
|
||||
|
||||
def get_redis_status(self):
|
||||
key = 'HEALTH_CHECK'
|
||||
|
||||
t1 = time.time()
|
||||
try:
|
||||
value = '1'
|
||||
cache.set(key, '1', 10)
|
||||
got = cache.get(key)
|
||||
t2 = time.time()
|
||||
if value == got:
|
||||
return True, t2 -t1
|
||||
return False, t2 -t1
|
||||
except:
|
||||
t2 = time.time()
|
||||
return False, t2 - t1
|
||||
|
||||
def get(self, request):
|
||||
redis_status, redis_time = self.get_redis_status()
|
||||
db_status, db_time = self.get_db_status()
|
||||
status = all([redis_status, db_status])
|
||||
data = {
|
||||
'status': status,
|
||||
'db_status': db_status,
|
||||
'db_time': db_time,
|
||||
'redis_status': redis_status,
|
||||
'redis_time': redis_time,
|
||||
'time': int(time.time())
|
||||
}
|
||||
return Response(data)
|
||||
|
||||
|
||||
class PrometheusMetricsApi(HealthApiMixin):
|
||||
permission_classes = (AllowAny,)
|
||||
|
||||
def get(self, request, *args, **kwargs):
|
||||
|
||||
@@ -143,6 +143,7 @@ class Config(dict):
|
||||
'REDIS_DB_SESSION': 5,
|
||||
'REDIS_DB_WS': 6,
|
||||
|
||||
'GLOBAL_ORG_DISPLAY_NAME': '',
|
||||
'SITE_URL': 'http://localhost:8080',
|
||||
'CAPTCHA_TEST_MODE': None,
|
||||
'TOKEN_EXPIRATION': 3600 * 24,
|
||||
@@ -267,7 +268,7 @@ class Config(dict):
|
||||
'WINDOWS_SSH_DEFAULT_SHELL': 'cmd',
|
||||
'FLOWER_URL': "127.0.0.1:5555",
|
||||
'DEFAULT_ORG_SHOW_ALL_USERS': True,
|
||||
'PERIOD_TASK_ENABLE': True,
|
||||
'PERIOD_TASK_ENABLED': True,
|
||||
'FORCE_SCRIPT_NAME': '',
|
||||
'LOGIN_CONFIRM_ENABLE': False,
|
||||
'WINDOWS_SKIP_ALL_MANUAL_PASSWORD': False,
|
||||
@@ -288,6 +289,7 @@ class Config(dict):
|
||||
'SESSION_SAVE_EVERY_REQUEST': True,
|
||||
'SESSION_EXPIRE_AT_BROWSER_CLOSE_FORCE': False,
|
||||
'FORGOT_PASSWORD_URL': '',
|
||||
'HEALTH_CHECK_TOKEN': ''
|
||||
}
|
||||
|
||||
def compatible_auth_openid_of_key(self):
|
||||
|
||||
@@ -120,3 +120,7 @@ CONNECTION_TOKEN_ENABLED = CONFIG.CONNECTION_TOKEN_ENABLED
|
||||
DISK_CHECK_ENABLED = CONFIG.DISK_CHECK_ENABLED
|
||||
FORGOT_PASSWORD_URL = CONFIG.FORGOT_PASSWORD_URL
|
||||
|
||||
|
||||
# 自定义默认组织名
|
||||
GLOBAL_ORG_DISPLAY_NAME = CONFIG.GLOBAL_ORG_DISPLAY_NAME
|
||||
HEALTH_CHECK_TOKEN = CONFIG.HEALTH_CHECK_TOKEN
|
||||
|
||||
@@ -48,7 +48,8 @@ urlpatterns = [
|
||||
path('', views.IndexView.as_view(), name='index'),
|
||||
path('api/v1/', include(api_v1)),
|
||||
re_path('api/(?P<app>\w+)/(?P<version>v\d)/.*', views.redirect_format_api),
|
||||
path('api/health/', views.HealthCheckView.as_view(), name="health"),
|
||||
path('api/health/', api.HealthCheckView.as_view(), name="health"),
|
||||
path('api/v1/health/', api.HealthCheckView.as_view(), name="health_v1"),
|
||||
# External apps url
|
||||
path('core/auth/captcha/', include('captcha.urls')),
|
||||
path('core/', include(app_view_patterns)),
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
from django.views.generic import TemplateView
|
||||
from django.shortcuts import redirect
|
||||
from common.permissions import PermissionsMixin, IsValidUser
|
||||
from common.permissions import IsValidUser
|
||||
from common.mixins.views import PermissionsMixin
|
||||
|
||||
__all__ = ['IndexView']
|
||||
|
||||
|
||||
@@ -17,7 +17,7 @@ from common.http import HttpResponseTemporaryRedirect
|
||||
|
||||
|
||||
__all__ = [
|
||||
'LunaView', 'I18NView', 'KokoView', 'WsView', 'HealthCheckView',
|
||||
'LunaView', 'I18NView', 'KokoView', 'WsView',
|
||||
'redirect_format_api', 'redirect_old_apps_view', 'UIView'
|
||||
]
|
||||
|
||||
@@ -64,13 +64,6 @@ def redirect_old_apps_view(request, *args, **kwargs):
|
||||
return HttpResponseTemporaryRedirect(new_path)
|
||||
|
||||
|
||||
class HealthCheckView(APIView):
|
||||
permission_classes = (AllowAny,)
|
||||
|
||||
def get(self, request):
|
||||
return JsonResponse({"status": 1, "time": int(time.time())})
|
||||
|
||||
|
||||
class WsView(APIView):
|
||||
ws_port = settings.HTTP_LISTEN_PORT + 1
|
||||
|
||||
|
||||
Binary file not shown.
File diff suppressed because it is too large
Load Diff
@@ -72,6 +72,7 @@ def create_or_update_celery_periodic_tasks(tasks):
|
||||
crontab=crontab,
|
||||
name=name,
|
||||
task=detail['task'],
|
||||
enabled=detail.get('enabled', True),
|
||||
args=json.dumps(detail.get('args', [])),
|
||||
kwargs=json.dumps(detail.get('kwargs', {})),
|
||||
description=detail.get('description') or ''
|
||||
|
||||
@@ -3,8 +3,8 @@
|
||||
from django.views.generic import TemplateView
|
||||
from django.conf import settings
|
||||
|
||||
from common.permissions import PermissionsMixin, IsOrgAdmin, IsOrgAuditor
|
||||
|
||||
from common.permissions import IsOrgAdmin, IsOrgAuditor
|
||||
from common.mixins.views import PermissionsMixin
|
||||
|
||||
__all__ = ['CeleryTaskLogView']
|
||||
|
||||
|
||||
@@ -90,6 +90,11 @@ class OrgMemberRelationBulkViewSet(JMSBulkRelationModelViewSet):
|
||||
filterset_class = OrgMemberRelationFilterSet
|
||||
search_fields = ('user__name', 'user__username', 'org__name')
|
||||
|
||||
def get_queryset(self):
|
||||
queryset = super().get_queryset()
|
||||
queryset = queryset.exclude(user__role=User.ROLE.APP)
|
||||
return queryset
|
||||
|
||||
def perform_bulk_destroy(self, queryset):
|
||||
objs = list(queryset.all().prefetch_related('user', 'org'))
|
||||
queryset.delete()
|
||||
|
||||
@@ -7,7 +7,7 @@ from django.db.models import signals
|
||||
from django.db.models import Q
|
||||
from django.utils.translation import ugettext_lazy as _
|
||||
|
||||
from common.utils import is_uuid, lazyproperty
|
||||
from common.utils import lazyproperty, settings
|
||||
from common.const import choices
|
||||
from common.db.models import ChoiceSet
|
||||
|
||||
@@ -193,7 +193,8 @@ class Organization(models.Model):
|
||||
|
||||
@classmethod
|
||||
def root(cls):
|
||||
return cls(id=cls.ROOT_ID, name=cls.ROOT_NAME)
|
||||
name = settings.GLOBAL_ORG_DISPLAY_NAME or cls.ROOT_NAME
|
||||
return cls(id=cls.ROOT_ID, name=name)
|
||||
|
||||
def is_root(self):
|
||||
return self.id == self.ROOT_ID
|
||||
|
||||
@@ -46,12 +46,19 @@ def subscribe_orgs_mapping_expire(sender, **kwargs):
|
||||
logger.debug("Start subscribe for expire orgs mapping from memory")
|
||||
|
||||
def keep_subscribe():
|
||||
subscribe = orgs_mapping_for_memory_pub_sub.subscribe()
|
||||
for message in subscribe.listen():
|
||||
if message['type'] != 'message':
|
||||
continue
|
||||
Organization.expire_orgs_mapping()
|
||||
logger.debug('Expire orgs mapping')
|
||||
while True:
|
||||
try:
|
||||
subscribe = orgs_mapping_for_memory_pub_sub.subscribe()
|
||||
for message in subscribe.listen():
|
||||
if message['type'] != 'message':
|
||||
continue
|
||||
if message['data'] == b'error':
|
||||
raise ValueError
|
||||
Organization.expire_orgs_mapping()
|
||||
logger.debug('Expire orgs mapping')
|
||||
except Exception as e:
|
||||
logger.exception(f'subscribe_orgs_mapping_expire: {e}')
|
||||
Organization.expire_orgs_mapping()
|
||||
|
||||
t = threading.Thread(target=keep_subscribe)
|
||||
t.daemon = True
|
||||
|
||||
@@ -20,3 +20,4 @@ class AssetPermissionViewSet(OrgBulkModelViewSet):
|
||||
model = AssetPermission
|
||||
serializer_class = serializers.AssetPermissionSerializer
|
||||
filterset_class = AssetPermissionFilter
|
||||
search_fields = ('name',)
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
from rest_framework import serializers
|
||||
from django.utils.translation import ugettext_lazy as _
|
||||
|
||||
from assets.models import Node, SystemUser, Asset
|
||||
from assets.models import Node, SystemUser, Asset, Platform
|
||||
from assets.serializers import ProtocolsField
|
||||
from perms.serializers.asset.permission import ActionsField
|
||||
|
||||
@@ -39,7 +39,9 @@ class AssetGrantedSerializer(serializers.ModelSerializer):
|
||||
被授权资产的数据结构
|
||||
"""
|
||||
protocols = ProtocolsField(label=_('Protocols'), required=False, read_only=True)
|
||||
platform = serializers.ReadOnlyField(source='platform_base')
|
||||
platform = serializers.SlugRelatedField(
|
||||
slug_field='name', queryset=Platform.objects.all(), label=_("Platform")
|
||||
)
|
||||
|
||||
class Meta:
|
||||
model = Asset
|
||||
|
||||
@@ -488,11 +488,12 @@ class UserGrantedAssetsQueryUtils(UserGrantedUtilsBase):
|
||||
|
||||
if granted_status == NodeFrom.granted:
|
||||
assets = Asset.objects.order_by().filter(nodes__id=node.id)
|
||||
return assets
|
||||
elif granted_status == NodeFrom.asset:
|
||||
return self._get_indirect_granted_node_assets(node.id)
|
||||
assets = self._get_indirect_granted_node_assets(node.id)
|
||||
else:
|
||||
return Asset.objects.none()
|
||||
assets = Asset.objects.none()
|
||||
assets = assets.order_by('hostname')
|
||||
return assets
|
||||
|
||||
def _get_indirect_granted_node_assets(self, id) -> AssetQuerySet:
|
||||
assets = Asset.objects.order_by().filter(nodes__id=id).distinct() & self.get_direct_granted_assets()
|
||||
@@ -538,6 +539,10 @@ class UserGrantedAssetsQueryUtils(UserGrantedUtilsBase):
|
||||
|
||||
|
||||
class UserGrantedNodesQueryUtils(UserGrantedUtilsBase):
|
||||
def sort(self, nodes):
|
||||
nodes = sorted(nodes, key=lambda x: x.value)
|
||||
return nodes
|
||||
|
||||
def get_node_children(self, key):
|
||||
if not key:
|
||||
return self.get_top_level_nodes()
|
||||
@@ -545,11 +550,13 @@ class UserGrantedNodesQueryUtils(UserGrantedUtilsBase):
|
||||
node = PermNode.objects.get(key=key)
|
||||
granted_status = node.get_granted_status(self.user)
|
||||
if granted_status == NodeFrom.granted:
|
||||
return PermNode.objects.filter(parent_key=key)
|
||||
nodes = PermNode.objects.filter(parent_key=key)
|
||||
elif granted_status in (NodeFrom.asset, NodeFrom.child):
|
||||
return self.get_indirect_granted_node_children(key)
|
||||
nodes = self.get_indirect_granted_node_children(key)
|
||||
else:
|
||||
return PermNode.objects.none()
|
||||
nodes = PermNode.objects.none()
|
||||
nodes = self.sort(nodes)
|
||||
return nodes
|
||||
|
||||
def get_indirect_granted_node_children(self, key):
|
||||
"""
|
||||
@@ -571,7 +578,8 @@ class UserGrantedNodesQueryUtils(UserGrantedUtilsBase):
|
||||
|
||||
def get_top_level_nodes(self):
|
||||
nodes = self.get_special_nodes()
|
||||
nodes.extend(self.get_indirect_granted_node_children(''))
|
||||
real_nodes = self.get_indirect_granted_node_children('')
|
||||
nodes.extend(self.sort(real_nodes))
|
||||
return nodes
|
||||
|
||||
def get_ungrouped_node(self):
|
||||
|
||||
@@ -24,6 +24,10 @@ class BasicSettingSerializer(serializers.Serializer):
|
||||
help_text=_('The forgot password url on login page, If you use '
|
||||
'ldap or cas external authentication, you can set it')
|
||||
)
|
||||
GLOBAL_ORG_DISPLAY_NAME = serializers.CharField(
|
||||
required=False, max_length=1024, allow_blank=True, allow_null=True, label=_("Global organization name"),
|
||||
help_text=_('The name of global organization to display')
|
||||
)
|
||||
|
||||
|
||||
class EmailSettingSerializer(serializers.Serializer):
|
||||
|
||||
@@ -6,6 +6,7 @@ import threading
|
||||
from django.dispatch import receiver
|
||||
from django.db.models.signals import post_save, pre_save
|
||||
from django.utils.functional import LazyObject
|
||||
from django.db import close_old_connections
|
||||
|
||||
from jumpserver.utils import current_request
|
||||
from common.decorator import on_transaction_commit
|
||||
@@ -71,13 +72,21 @@ def subscribe_settings_change(sender, **kwargs):
|
||||
logger.debug("Start subscribe setting change")
|
||||
|
||||
def keep_subscribe():
|
||||
sub = setting_pub_sub.subscribe()
|
||||
for msg in sub.listen():
|
||||
if msg["type"] != "message":
|
||||
continue
|
||||
item = msg['data'].decode()
|
||||
logger.debug("Found setting change: {}".format(str(item)))
|
||||
Setting.refresh_item(item)
|
||||
while True:
|
||||
try:
|
||||
sub = setting_pub_sub.subscribe()
|
||||
for msg in sub.listen():
|
||||
close_old_connections()
|
||||
if msg["type"] != "message":
|
||||
continue
|
||||
item = msg['data'].decode()
|
||||
logger.debug("Found setting change: {}".format(str(item)))
|
||||
Setting.refresh_item(item)
|
||||
except Exception as e:
|
||||
logger.exception(f'subscribe_settings_change: {e}')
|
||||
close_old_connections()
|
||||
Setting.refresh_all_settings()
|
||||
|
||||
t = threading.Thread(target=keep_subscribe)
|
||||
t.daemon = True
|
||||
t.start()
|
||||
|
||||
@@ -5,4 +5,4 @@ from .session import *
|
||||
from .command import *
|
||||
from .task import *
|
||||
from .storage import *
|
||||
from .component import *
|
||||
from .status import *
|
||||
|
||||
@@ -11,6 +11,7 @@ from rest_framework.response import Response
|
||||
from rest_framework.decorators import action
|
||||
from django.template import loader
|
||||
|
||||
from common.http import is_true
|
||||
from terminal.models import CommandStorage, Command
|
||||
from terminal.filters import CommandFilter
|
||||
from orgs.utils import current_org
|
||||
@@ -140,7 +141,19 @@ class CommandViewSet(viewsets.ModelViewSet):
|
||||
if session_id and not command_storage_id:
|
||||
# 会话里的命令列表肯定会提供 session_id,这里防止 merge 的时候取全量的数据
|
||||
return self.merge_all_storage_list(request, *args, **kwargs)
|
||||
return super().list(request, *args, **kwargs)
|
||||
|
||||
queryset = self.filter_queryset(self.get_queryset())
|
||||
|
||||
page = self.paginate_queryset(queryset)
|
||||
if page is not None:
|
||||
serializer = self.get_serializer(page, many=True)
|
||||
return self.get_paginated_response(serializer.data)
|
||||
|
||||
# 适配像 ES 这种没有指定分页只返回少量数据的情况
|
||||
queryset = queryset[:]
|
||||
|
||||
serializer = self.get_serializer(queryset, many=True)
|
||||
return Response(serializer.data)
|
||||
|
||||
def get_queryset(self):
|
||||
command_storage_id = self.request.query_params.get('command_storage_id')
|
||||
|
||||
@@ -1,34 +0,0 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
#
|
||||
|
||||
import logging
|
||||
from rest_framework import generics, status
|
||||
from rest_framework.views import Response
|
||||
|
||||
from .. import serializers
|
||||
from ..utils import ComponentsMetricsUtil
|
||||
from common.permissions import IsAppUser, IsSuperUser
|
||||
|
||||
logger = logging.getLogger(__file__)
|
||||
|
||||
|
||||
__all__ = [
|
||||
'ComponentsStateAPIView', 'ComponentsMetricsAPIView',
|
||||
]
|
||||
|
||||
|
||||
class ComponentsStateAPIView(generics.CreateAPIView):
|
||||
""" koko, guacamole, omnidb 上报状态 """
|
||||
permission_classes = (IsAppUser,)
|
||||
serializer_class = serializers.ComponentsStateSerializer
|
||||
|
||||
|
||||
class ComponentsMetricsAPIView(generics.GenericAPIView):
|
||||
""" 返回汇总组件指标数据 """
|
||||
permission_classes = (IsSuperUser,)
|
||||
|
||||
def get(self, request, *args, **kwargs):
|
||||
tp = request.query_params.get('type')
|
||||
util = ComponentsMetricsUtil()
|
||||
metrics = util.get_metrics(tp)
|
||||
return Response(metrics, status=status.HTTP_200_OK)
|
||||
74
apps/terminal/api/status.py
Normal file
74
apps/terminal/api/status.py
Normal file
@@ -0,0 +1,74 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
#
|
||||
|
||||
import logging
|
||||
from django.shortcuts import get_object_or_404
|
||||
from rest_framework import viewsets, generics
|
||||
from rest_framework.views import Response
|
||||
from rest_framework import status
|
||||
|
||||
from common.permissions import IsAppUser, IsOrgAdminOrAppUser, IsSuperUser
|
||||
from ..models import Terminal, Status, Session
|
||||
from .. import serializers
|
||||
from ..utils import TypedComponentsStatusMetricsUtil
|
||||
|
||||
logger = logging.getLogger(__file__)
|
||||
|
||||
|
||||
__all__ = [
|
||||
'StatusViewSet',
|
||||
'ComponentsMetricsAPIView',
|
||||
]
|
||||
|
||||
|
||||
class StatusViewSet(viewsets.ModelViewSet):
|
||||
queryset = Status.objects.all()
|
||||
serializer_class = serializers.StatusSerializer
|
||||
permission_classes = (IsOrgAdminOrAppUser,)
|
||||
session_serializer_class = serializers.SessionSerializer
|
||||
task_serializer_class = serializers.TaskSerializer
|
||||
|
||||
def create(self, request, *args, **kwargs):
|
||||
serializer = self.get_serializer(data=request.data)
|
||||
serializer.is_valid(raise_exception=True)
|
||||
self.handle_sessions()
|
||||
self.perform_create(serializer)
|
||||
tasks = self.request.user.terminal.task_set.filter(is_finished=False)
|
||||
serializer = self.task_serializer_class(tasks, many=True)
|
||||
return Response(serializer.data, status=201)
|
||||
|
||||
def handle_sessions(self):
|
||||
session_ids = self.request.data.get('sessions', [])
|
||||
# guacamole 上报的 session 是字符串
|
||||
# "[53cd3e47-210f-41d8-b3c6-a184f3, 53cd3e47-210f-41d8-b3c6-a184f4]"
|
||||
if isinstance(session_ids, str):
|
||||
session_ids = session_ids[1:-1].split(',')
|
||||
session_ids = [sid.strip() for sid in session_ids if sid.strip()]
|
||||
Session.set_sessions_active(session_ids)
|
||||
|
||||
def get_queryset(self):
|
||||
terminal_id = self.kwargs.get("terminal", None)
|
||||
if terminal_id:
|
||||
terminal = get_object_or_404(Terminal, id=terminal_id)
|
||||
return terminal.status_set.all()
|
||||
return super().get_queryset()
|
||||
|
||||
def perform_create(self, serializer):
|
||||
serializer.validated_data.pop('sessions', None)
|
||||
serializer.validated_data["terminal"] = self.request.user.terminal
|
||||
return super().perform_create(serializer)
|
||||
|
||||
def get_permissions(self):
|
||||
if self.action == "create":
|
||||
self.permission_classes = (IsAppUser,)
|
||||
return super().get_permissions()
|
||||
|
||||
|
||||
class ComponentsMetricsAPIView(generics.GenericAPIView):
|
||||
""" 返回汇总组件指标数据 """
|
||||
permission_classes = (IsSuperUser,)
|
||||
|
||||
def get(self, request, *args, **kwargs):
|
||||
util = TypedComponentsStatusMetricsUtil()
|
||||
metrics = util.get_metrics()
|
||||
return Response(metrics, status=status.HTTP_200_OK)
|
||||
@@ -4,8 +4,7 @@ import logging
|
||||
import uuid
|
||||
|
||||
from django.core.cache import cache
|
||||
from django.shortcuts import get_object_or_404
|
||||
from rest_framework import viewsets, generics
|
||||
from rest_framework import generics
|
||||
from rest_framework.views import APIView, Response
|
||||
from rest_framework import status
|
||||
from django.conf import settings
|
||||
@@ -13,13 +12,13 @@ from django.conf import settings
|
||||
|
||||
from common.drf.api import JMSBulkModelViewSet
|
||||
from common.utils import get_object_or_none
|
||||
from common.permissions import IsAppUser, IsOrgAdminOrAppUser, IsSuperUser, WithBootstrapToken
|
||||
from ..models import Terminal, Status, Session
|
||||
from common.permissions import IsAppUser, IsSuperUser, WithBootstrapToken
|
||||
from ..models import Terminal
|
||||
from .. import serializers
|
||||
from .. import exceptions
|
||||
|
||||
__all__ = [
|
||||
'TerminalViewSet', 'StatusViewSet', 'TerminalConfig',
|
||||
'TerminalViewSet', 'TerminalConfig',
|
||||
'TerminalRegistrationApi',
|
||||
]
|
||||
logger = logging.getLogger(__file__)
|
||||
@@ -72,45 +71,6 @@ class TerminalViewSet(JMSBulkModelViewSet):
|
||||
return queryset
|
||||
|
||||
|
||||
class StatusViewSet(viewsets.ModelViewSet):
|
||||
queryset = Status.objects.all()
|
||||
serializer_class = serializers.StatusSerializer
|
||||
permission_classes = (IsOrgAdminOrAppUser,)
|
||||
session_serializer_class = serializers.SessionSerializer
|
||||
task_serializer_class = serializers.TaskSerializer
|
||||
|
||||
def create(self, request, *args, **kwargs):
|
||||
self.handle_sessions()
|
||||
tasks = self.request.user.terminal.task_set.filter(is_finished=False)
|
||||
serializer = self.task_serializer_class(tasks, many=True)
|
||||
return Response(serializer.data, status=201)
|
||||
|
||||
def handle_sessions(self):
|
||||
session_ids = self.request.data.get('sessions', [])
|
||||
# guacamole 上报的 session 是字符串
|
||||
# "[53cd3e47-210f-41d8-b3c6-a184f3, 53cd3e47-210f-41d8-b3c6-a184f4]"
|
||||
if isinstance(session_ids, str):
|
||||
session_ids = session_ids[1:-1].split(',')
|
||||
session_ids = [sid.strip() for sid in session_ids if sid.strip()]
|
||||
Session.set_sessions_active(session_ids)
|
||||
|
||||
def get_queryset(self):
|
||||
terminal_id = self.kwargs.get("terminal", None)
|
||||
if terminal_id:
|
||||
terminal = get_object_or_404(Terminal, id=terminal_id)
|
||||
self.queryset = terminal.status_set.all()
|
||||
return self.queryset
|
||||
|
||||
def perform_create(self, serializer):
|
||||
serializer.validated_data["terminal"] = self.request.user.terminal
|
||||
return super().perform_create(serializer)
|
||||
|
||||
def get_permissions(self):
|
||||
if self.action == "create":
|
||||
self.permission_classes = (IsAppUser,)
|
||||
return super().get_permissions()
|
||||
|
||||
|
||||
class TerminalConfig(APIView):
|
||||
permission_classes = (IsAppUser,)
|
||||
|
||||
|
||||
@@ -10,6 +10,7 @@ import inspect
|
||||
from django.db.models import QuerySet as DJQuerySet
|
||||
from elasticsearch import Elasticsearch
|
||||
from elasticsearch.helpers import bulk
|
||||
from elasticsearch.exceptions import RequestError
|
||||
|
||||
from common.utils.common import lazyproperty
|
||||
from common.utils import get_logger
|
||||
@@ -25,8 +26,21 @@ class CommandStore():
|
||||
kwargs = config.get("OTHER", {})
|
||||
self.index = config.get("INDEX") or 'jumpserver'
|
||||
self.doc_type = config.get("DOC_TYPE") or 'command_store'
|
||||
|
||||
ignore_verify_certs = kwargs.pop('ignore_verify_certs', False)
|
||||
if ignore_verify_certs:
|
||||
kwargs['verify_certs'] = None
|
||||
self.es = Elasticsearch(hosts=hosts, max_retries=0, **kwargs)
|
||||
|
||||
def pre_use_check(self):
|
||||
self._ensure_index_exists()
|
||||
|
||||
def _ensure_index_exists(self):
|
||||
try:
|
||||
self.es.indices.create(self.index)
|
||||
except RequestError:
|
||||
pass
|
||||
|
||||
@staticmethod
|
||||
def make_data(command):
|
||||
data = dict(
|
||||
@@ -230,6 +244,7 @@ class QuerySet(DJQuerySet):
|
||||
uqs = QuerySet(self._command_store_config)
|
||||
uqs._method_calls = self._method_calls.copy()
|
||||
uqs._slice = self._slice
|
||||
uqs.model = self.model
|
||||
return uqs
|
||||
|
||||
def count(self, limit_to_max_result_window=True):
|
||||
|
||||
@@ -31,6 +31,7 @@ class ComponentStatusChoices(TextChoices):
|
||||
critical = 'critical', _('Critical')
|
||||
high = 'high', _('High')
|
||||
normal = 'normal', _('Normal')
|
||||
offline = 'offline', _('Offline')
|
||||
|
||||
@classmethod
|
||||
def status(cls):
|
||||
|
||||
18
apps/terminal/migrations/0033_auto_20210324_1008.py
Normal file
18
apps/terminal/migrations/0033_auto_20210324_1008.py
Normal file
@@ -0,0 +1,18 @@
|
||||
# Generated by Django 3.1 on 2021-03-24 02:08
|
||||
|
||||
from django.db import migrations, models
|
||||
|
||||
|
||||
class Migration(migrations.Migration):
|
||||
|
||||
dependencies = [
|
||||
('terminal', '0032_auto_20210302_1853'),
|
||||
]
|
||||
|
||||
operations = [
|
||||
migrations.AlterField(
|
||||
model_name='session',
|
||||
name='login_from',
|
||||
field=models.CharField(choices=[('ST', 'SSH Terminal'), ('RT', 'RDP Terminal'), ('WT', 'Web Terminal')], default='ST', max_length=2, verbose_name='Login from'),
|
||||
),
|
||||
]
|
||||
42
apps/terminal/migrations/0034_auto_20210406_1434.py
Normal file
42
apps/terminal/migrations/0034_auto_20210406_1434.py
Normal file
@@ -0,0 +1,42 @@
|
||||
# Generated by Django 3.1 on 2021-04-06 06:34
|
||||
|
||||
from django.db import migrations, models
|
||||
|
||||
|
||||
class Migration(migrations.Migration):
|
||||
|
||||
dependencies = [
|
||||
('terminal', '0033_auto_20210324_1008'),
|
||||
]
|
||||
|
||||
operations = [
|
||||
migrations.RemoveField(
|
||||
model_name='status',
|
||||
name='cpu_used',
|
||||
),
|
||||
migrations.AddField(
|
||||
model_name='status',
|
||||
name='cpu_load',
|
||||
field=models.FloatField(default=0, verbose_name='CPU Load'),
|
||||
),
|
||||
migrations.AddField(
|
||||
model_name='status',
|
||||
name='disk_used',
|
||||
field=models.FloatField(default=0, verbose_name='Disk Used'),
|
||||
),
|
||||
migrations.AlterField(
|
||||
model_name='status',
|
||||
name='boot_time',
|
||||
field=models.FloatField(default=0, verbose_name='Boot Time'),
|
||||
),
|
||||
migrations.AlterField(
|
||||
model_name='status',
|
||||
name='connections',
|
||||
field=models.IntegerField(default=0, verbose_name='Connections'),
|
||||
),
|
||||
migrations.AlterField(
|
||||
model_name='status',
|
||||
name='threads',
|
||||
field=models.IntegerField(default=0, verbose_name='Threads'),
|
||||
),
|
||||
]
|
||||
@@ -14,12 +14,12 @@ from assets.models import Asset
|
||||
from orgs.mixins.models import OrgModelMixin
|
||||
from common.db.models import ChoiceSet
|
||||
from ..backends import get_multi_command_storage
|
||||
from .terminal import Terminal
|
||||
|
||||
|
||||
class Session(OrgModelMixin):
|
||||
class LOGIN_FROM(ChoiceSet):
|
||||
ST = 'ST', 'SSH Terminal'
|
||||
RT = 'RT', 'RDP Terminal'
|
||||
WT = 'WT', 'Web Terminal'
|
||||
|
||||
class PROTOCOL(ChoiceSet):
|
||||
@@ -46,7 +46,7 @@ class Session(OrgModelMixin):
|
||||
is_finished = models.BooleanField(default=False, db_index=True)
|
||||
has_replay = models.BooleanField(default=False, verbose_name=_("Replay"))
|
||||
has_command = models.BooleanField(default=False, verbose_name=_("Command"))
|
||||
terminal = models.ForeignKey(Terminal, null=True, on_delete=models.DO_NOTHING, db_constraint=False)
|
||||
terminal = models.ForeignKey('terminal.Terminal', null=True, on_delete=models.DO_NOTHING, db_constraint=False)
|
||||
protocol = models.CharField(choices=PROTOCOL.choices, default='ssh', max_length=16, db_index=True)
|
||||
date_start = models.DateTimeField(verbose_name=_("Date start"), db_index=True, default=timezone.now)
|
||||
date_end = models.DateTimeField(verbose_name=_("Date end"), null=True)
|
||||
|
||||
@@ -3,26 +3,62 @@ from __future__ import unicode_literals
|
||||
import uuid
|
||||
|
||||
from django.db import models
|
||||
from django.forms.models import model_to_dict
|
||||
from django.core.cache import cache
|
||||
from django.utils.translation import ugettext_lazy as _
|
||||
|
||||
from .terminal import Terminal
|
||||
from common.utils import get_logger
|
||||
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
class Status(models.Model):
|
||||
id = models.UUIDField(default=uuid.uuid4, primary_key=True)
|
||||
session_online = models.IntegerField(verbose_name=_("Session Online"), default=0)
|
||||
cpu_used = models.FloatField(verbose_name=_("CPU Usage"))
|
||||
cpu_load = models.FloatField(verbose_name=_("CPU Load"), default=0)
|
||||
memory_used = models.FloatField(verbose_name=_("Memory Used"))
|
||||
connections = models.IntegerField(verbose_name=_("Connections"))
|
||||
threads = models.IntegerField(verbose_name=_("Threads"))
|
||||
boot_time = models.FloatField(verbose_name=_("Boot Time"))
|
||||
terminal = models.ForeignKey(Terminal, null=True, on_delete=models.CASCADE)
|
||||
disk_used = models.FloatField(verbose_name=_("Disk Used"), default=0)
|
||||
connections = models.IntegerField(verbose_name=_("Connections"), default=0)
|
||||
threads = models.IntegerField(verbose_name=_("Threads"), default=0)
|
||||
boot_time = models.FloatField(verbose_name=_("Boot Time"), default=0)
|
||||
terminal = models.ForeignKey('terminal.Terminal', null=True, on_delete=models.CASCADE)
|
||||
date_created = models.DateTimeField(auto_now_add=True)
|
||||
|
||||
CACHE_KEY = 'TERMINAL_STATUS_{}'
|
||||
|
||||
class Meta:
|
||||
db_table = 'terminal_status'
|
||||
get_latest_by = 'date_created'
|
||||
|
||||
def __str__(self):
|
||||
return self.date_created.strftime("%Y-%m-%d %H:%M:%S")
|
||||
def save_to_cache(self):
|
||||
if not self.terminal:
|
||||
return
|
||||
key = self.CACHE_KEY.format(self.terminal.id)
|
||||
data = model_to_dict(self)
|
||||
cache.set(key, data, 60*3)
|
||||
return data
|
||||
|
||||
@classmethod
|
||||
def get_terminal_latest_status(cls, terminal):
|
||||
from ..utils import ComputeStatUtil
|
||||
stat = cls.get_terminal_latest_stat(terminal)
|
||||
return ComputeStatUtil.compute_component_status(stat)
|
||||
|
||||
@classmethod
|
||||
def get_terminal_latest_stat(cls, terminal):
|
||||
key = cls.CACHE_KEY.format(terminal.id)
|
||||
data = cache.get(key)
|
||||
if not data:
|
||||
return None
|
||||
data.pop('terminal', None)
|
||||
stat = cls(**data)
|
||||
stat.terminal = terminal
|
||||
return stat
|
||||
|
||||
def save(self, force_insert=False, force_update=False, using=None,
|
||||
update_fields=None):
|
||||
self.terminal.set_alive(ttl=120)
|
||||
return self.save_to_cache()
|
||||
# return super().save()
|
||||
|
||||
|
||||
@@ -76,6 +76,15 @@ class CommandStorage(CommonModelMixin):
|
||||
qs.model = Command
|
||||
return qs
|
||||
|
||||
def save(self, force_insert=False, force_update=False, using=None,
|
||||
update_fields=None):
|
||||
super().save()
|
||||
|
||||
if self.type in TYPE_ENGINE_MAPPING:
|
||||
engine_mod = import_module(TYPE_ENGINE_MAPPING[self.type])
|
||||
backend = engine_mod.CommandStore(self.config)
|
||||
backend.pre_use_check()
|
||||
|
||||
|
||||
class ReplayStorage(CommonModelMixin):
|
||||
name = models.CharField(max_length=128, verbose_name=_("Name"), unique=True)
|
||||
|
||||
@@ -1,168 +1,64 @@
|
||||
from __future__ import unicode_literals
|
||||
import uuid
|
||||
|
||||
from django.db import models
|
||||
from django.core.cache import cache
|
||||
from django.utils.translation import ugettext_lazy as _
|
||||
from django.conf import settings
|
||||
from django.core.cache import cache
|
||||
|
||||
from common.utils import get_logger
|
||||
from users.models import User
|
||||
from orgs.utils import tmp_to_root_org
|
||||
from .status import Status
|
||||
from .. import const
|
||||
from ..const import ComponentStatusChoices as StatusChoice
|
||||
from .session import Session
|
||||
|
||||
|
||||
logger = get_logger(__file__)
|
||||
|
||||
|
||||
class ComputeStatusMixin:
|
||||
|
||||
# system status
|
||||
@staticmethod
|
||||
def _common_compute_system_status(value, thresholds):
|
||||
if thresholds[0] <= value <= thresholds[1]:
|
||||
return const.ComponentStatusChoices.normal.value
|
||||
elif thresholds[1] < value <= thresholds[2]:
|
||||
return const.ComponentStatusChoices.high.value
|
||||
else:
|
||||
return const.ComponentStatusChoices.critical.value
|
||||
|
||||
def _compute_system_cpu_load_1_status(self, value):
|
||||
thresholds = [0, 5, 20]
|
||||
return self._common_compute_system_status(value, thresholds)
|
||||
|
||||
def _compute_system_memory_used_percent_status(self, value):
|
||||
thresholds = [0, 85, 95]
|
||||
return self._common_compute_system_status(value, thresholds)
|
||||
|
||||
def _compute_system_disk_used_percent_status(self, value):
|
||||
thresholds = [0, 80, 99]
|
||||
return self._common_compute_system_status(value, thresholds)
|
||||
|
||||
def _compute_system_status(self, state):
|
||||
system_status_keys = [
|
||||
'system_cpu_load_1', 'system_memory_used_percent', 'system_disk_used_percent'
|
||||
]
|
||||
system_status = []
|
||||
for system_status_key in system_status_keys:
|
||||
state_value = state.get(system_status_key)
|
||||
if state_value is None:
|
||||
msg = 'state: {}, state_key: {}, state_value: {}'
|
||||
logger.debug(msg.format(state, system_status_key, state_value))
|
||||
state_value = 0
|
||||
status = getattr(self, f'_compute_{system_status_key}_status')(state_value)
|
||||
system_status.append(status)
|
||||
return system_status
|
||||
|
||||
def _compute_component_status(self, state):
|
||||
system_status = self._compute_system_status(state)
|
||||
if const.ComponentStatusChoices.critical in system_status:
|
||||
return const.ComponentStatusChoices.critical
|
||||
elif const.ComponentStatusChoices.high in system_status:
|
||||
return const.ComponentStatusChoices.high
|
||||
else:
|
||||
return const.ComponentStatusChoices.normal
|
||||
|
||||
@staticmethod
|
||||
def _compute_component_status_display(status):
|
||||
return getattr(const.ComponentStatusChoices, status).label
|
||||
|
||||
|
||||
class TerminalStateMixin(ComputeStatusMixin):
|
||||
CACHE_KEY_COMPONENT_STATE = 'CACHE_KEY_COMPONENT_STATE_TERMINAL_{}'
|
||||
CACHE_TIMEOUT = 120
|
||||
class TerminalStatusMixin:
|
||||
ALIVE_KEY = 'TERMINAL_ALIVE_{}'
|
||||
id: str
|
||||
|
||||
@property
|
||||
def cache_key(self):
|
||||
return self.CACHE_KEY_COMPONENT_STATE.format(str(self.id))
|
||||
|
||||
# get
|
||||
def _get_from_cache(self):
|
||||
return cache.get(self.cache_key)
|
||||
|
||||
def _set_to_cache(self, state):
|
||||
cache.set(self.cache_key, state, self.CACHE_TIMEOUT)
|
||||
|
||||
# set
|
||||
def _add_status(self, state):
|
||||
status = self._compute_component_status(state)
|
||||
status_display = self._compute_component_status_display(status)
|
||||
state.update({
|
||||
'status': status,
|
||||
'status_display': status_display
|
||||
})
|
||||
def latest_status(self):
|
||||
return Status.get_terminal_latest_status(self)
|
||||
|
||||
@property
|
||||
def state(self):
|
||||
state = self._get_from_cache()
|
||||
return state or {}
|
||||
|
||||
@state.setter
|
||||
def state(self, state):
|
||||
self._add_status(state)
|
||||
self._set_to_cache(state)
|
||||
|
||||
|
||||
class TerminalStatusMixin(TerminalStateMixin):
|
||||
|
||||
# alive
|
||||
@property
|
||||
def is_alive(self):
|
||||
return bool(self.state)
|
||||
|
||||
# status
|
||||
@property
|
||||
def status(self):
|
||||
if self.is_alive:
|
||||
return self.state['status']
|
||||
else:
|
||||
return const.ComponentStatusChoices.critical.value
|
||||
def latest_status_display(self):
|
||||
return self.latest_status.label
|
||||
|
||||
@property
|
||||
def status_display(self):
|
||||
return self._compute_component_status_display(self.status)
|
||||
def latest_stat(self):
|
||||
return Status.get_terminal_latest_stat(self)
|
||||
|
||||
@property
|
||||
def is_normal(self):
|
||||
return self.status == const.ComponentStatusChoices.normal.value
|
||||
return self.latest_status == StatusChoice.normal
|
||||
|
||||
@property
|
||||
def is_high(self):
|
||||
return self.status == const.ComponentStatusChoices.high.value
|
||||
return self.latest_status == StatusChoice.high
|
||||
|
||||
@property
|
||||
def is_critical(self):
|
||||
return self.status == const.ComponentStatusChoices.critical.value
|
||||
|
||||
|
||||
class Terminal(TerminalStatusMixin, models.Model):
|
||||
id = models.UUIDField(default=uuid.uuid4, primary_key=True)
|
||||
name = models.CharField(max_length=128, verbose_name=_('Name'))
|
||||
type = models.CharField(
|
||||
choices=const.TerminalTypeChoices.choices, default=const.TerminalTypeChoices.koko.value,
|
||||
max_length=64, verbose_name=_('type')
|
||||
)
|
||||
remote_addr = models.CharField(max_length=128, blank=True, verbose_name=_('Remote Address'))
|
||||
ssh_port = models.IntegerField(verbose_name=_('SSH Port'), default=2222)
|
||||
http_port = models.IntegerField(verbose_name=_('HTTP Port'), default=5000)
|
||||
command_storage = models.CharField(max_length=128, verbose_name=_("Command storage"), default='default')
|
||||
replay_storage = models.CharField(max_length=128, verbose_name=_("Replay storage"), default='default')
|
||||
user = models.OneToOneField(User, related_name='terminal', verbose_name='Application User', null=True, on_delete=models.CASCADE)
|
||||
is_accepted = models.BooleanField(default=False, verbose_name='Is Accepted')
|
||||
is_deleted = models.BooleanField(default=False)
|
||||
date_created = models.DateTimeField(auto_now_add=True)
|
||||
comment = models.TextField(blank=True, verbose_name=_('Comment'))
|
||||
return self.latest_status == StatusChoice.critical
|
||||
|
||||
@property
|
||||
def is_active(self):
|
||||
if self.user and self.user.is_active:
|
||||
return True
|
||||
return False
|
||||
def is_alive(self):
|
||||
key = self.ALIVE_KEY.format(self.id)
|
||||
# return self.latest_status != StatusChoice.offline
|
||||
return cache.get(key, False)
|
||||
|
||||
@is_active.setter
|
||||
def is_active(self, active):
|
||||
if self.user:
|
||||
self.user.is_active = active
|
||||
self.user.save()
|
||||
def set_alive(self, ttl=120):
|
||||
key = self.ALIVE_KEY.format(self.id)
|
||||
cache.set(key, True, ttl)
|
||||
|
||||
|
||||
class StorageMixin:
|
||||
command_storage: str
|
||||
replay_storage: str
|
||||
|
||||
def get_command_storage(self):
|
||||
from .storage import CommandStorage
|
||||
@@ -198,6 +94,44 @@ class Terminal(TerminalStatusMixin, models.Model):
|
||||
config = self.get_replay_storage_config()
|
||||
return {"TERMINAL_REPLAY_STORAGE": config}
|
||||
|
||||
|
||||
class Terminal(StorageMixin, TerminalStatusMixin, models.Model):
|
||||
id = models.UUIDField(default=uuid.uuid4, primary_key=True)
|
||||
name = models.CharField(max_length=128, verbose_name=_('Name'))
|
||||
type = models.CharField(
|
||||
choices=const.TerminalTypeChoices.choices, default=const.TerminalTypeChoices.koko.value,
|
||||
max_length=64, verbose_name=_('type')
|
||||
)
|
||||
remote_addr = models.CharField(max_length=128, blank=True, verbose_name=_('Remote Address'))
|
||||
ssh_port = models.IntegerField(verbose_name=_('SSH Port'), default=2222)
|
||||
http_port = models.IntegerField(verbose_name=_('HTTP Port'), default=5000)
|
||||
command_storage = models.CharField(max_length=128, verbose_name=_("Command storage"), default='default')
|
||||
replay_storage = models.CharField(max_length=128, verbose_name=_("Replay storage"), default='default')
|
||||
user = models.OneToOneField(User, related_name='terminal', verbose_name='Application User', null=True, on_delete=models.CASCADE)
|
||||
is_accepted = models.BooleanField(default=False, verbose_name='Is Accepted')
|
||||
is_deleted = models.BooleanField(default=False)
|
||||
date_created = models.DateTimeField(auto_now_add=True)
|
||||
comment = models.TextField(blank=True, verbose_name=_('Comment'))
|
||||
|
||||
@property
|
||||
def is_active(self):
|
||||
if self.user and self.user.is_active:
|
||||
return True
|
||||
return False
|
||||
|
||||
@is_active.setter
|
||||
def is_active(self, active):
|
||||
if self.user:
|
||||
self.user.is_active = active
|
||||
self.user.save()
|
||||
|
||||
def get_online_sessions(self):
|
||||
with tmp_to_root_org():
|
||||
return Session.objects.filter(terminal=self, is_finished=False)
|
||||
|
||||
def get_online_session_count(self):
|
||||
return self.get_online_sessions().count()
|
||||
|
||||
@staticmethod
|
||||
def get_login_title_setting():
|
||||
login_title = None
|
||||
|
||||
@@ -4,4 +4,3 @@ from .terminal import *
|
||||
from .session import *
|
||||
from .storage import *
|
||||
from .command import *
|
||||
from .components import *
|
||||
|
||||
@@ -1,25 +0,0 @@
|
||||
|
||||
from rest_framework import serializers
|
||||
from django.utils.translation import ugettext_lazy as _
|
||||
|
||||
|
||||
class ComponentsStateSerializer(serializers.Serializer):
|
||||
# system
|
||||
system_cpu_load_1 = serializers.FloatField(
|
||||
required=False, label=_("System cpu load (1 minutes)")
|
||||
)
|
||||
system_memory_used_percent = serializers.FloatField(
|
||||
required=False, label=_('System memory used percent')
|
||||
)
|
||||
system_disk_used_percent = serializers.FloatField(
|
||||
required=False, label=_('System disk used percent')
|
||||
)
|
||||
# sessions
|
||||
session_active_count = serializers.IntegerField(
|
||||
required=False, label=_("Session active count")
|
||||
)
|
||||
|
||||
def save(self, **kwargs):
|
||||
request = self.context['request']
|
||||
terminal = request.user.terminal
|
||||
terminal.state = self.validated_data
|
||||
@@ -181,7 +181,10 @@ class CommandStorageTypeESSerializer(serializers.Serializer):
|
||||
max_length=1024, default='jumpserver', label=_('Index'), allow_null=True
|
||||
)
|
||||
DOC_TYPE = ReadableHiddenField(default='command', label=_('Doc type'), allow_null=True)
|
||||
|
||||
ignore_verify_certs = serializers.BooleanField(
|
||||
default=False, label=_('Ignore Certificate Verification'),
|
||||
source='OTHER.ignore_verify_certs', allow_null=True,
|
||||
)
|
||||
|
||||
# mapping
|
||||
|
||||
|
||||
@@ -9,15 +9,34 @@ from common.utils import get_request_ip
|
||||
from ..models import (
|
||||
Terminal, Status, Session, Task, CommandStorage, ReplayStorage
|
||||
)
|
||||
from .components import ComponentsStateSerializer
|
||||
|
||||
|
||||
class StatusSerializer(serializers.ModelSerializer):
|
||||
sessions = serializers.ListSerializer(
|
||||
child=serializers.CharField(max_length=36), write_only=True
|
||||
)
|
||||
|
||||
class Meta:
|
||||
fields = [
|
||||
'id',
|
||||
'cpu_load', 'memory_used', 'disk_used',
|
||||
'session_online', 'sessions',
|
||||
'terminal', 'date_created',
|
||||
]
|
||||
extra_kwargs = {
|
||||
"cpu_load": {'default': 0},
|
||||
"memory_used": {'default': 0},
|
||||
"disk_used": {'default': 0},
|
||||
}
|
||||
model = Status
|
||||
|
||||
|
||||
class TerminalSerializer(BulkModelSerializer):
|
||||
session_online = serializers.SerializerMethodField()
|
||||
session_online = serializers.ReadOnlyField(source='get_online_session_count')
|
||||
is_alive = serializers.BooleanField(read_only=True)
|
||||
status = serializers.CharField(read_only=True)
|
||||
status_display = serializers.CharField(read_only=True)
|
||||
state = ComponentsStateSerializer(read_only=True)
|
||||
status = serializers.CharField(read_only=True, source='latest_status')
|
||||
status_display = serializers.CharField(read_only=True, source='latest_status_display')
|
||||
stat = StatusSerializer(read_only=True, source='latest_stat')
|
||||
|
||||
class Meta:
|
||||
model = Terminal
|
||||
@@ -25,7 +44,7 @@ class TerminalSerializer(BulkModelSerializer):
|
||||
'id', 'name', 'type', 'remote_addr', 'http_port', 'ssh_port',
|
||||
'comment', 'is_accepted', "is_active", 'session_online',
|
||||
'is_alive', 'date_created', 'command_storage', 'replay_storage',
|
||||
'status', 'status_display', 'state'
|
||||
'status', 'status_display', 'stat'
|
||||
]
|
||||
read_only_fields = ['type', 'date_created']
|
||||
|
||||
@@ -54,16 +73,6 @@ class TerminalSerializer(BulkModelSerializer):
|
||||
else:
|
||||
raise serializers.ValidationError(_('Not found'))
|
||||
|
||||
@staticmethod
|
||||
def get_session_online(obj):
|
||||
return Session.objects.filter(terminal=obj, is_finished=False).count()
|
||||
|
||||
|
||||
class StatusSerializer(serializers.ModelSerializer):
|
||||
class Meta:
|
||||
fields = ['id', 'terminal']
|
||||
model = Status
|
||||
|
||||
|
||||
class TaskSerializer(BulkModelSerializer):
|
||||
class Meta:
|
||||
|
||||
@@ -29,7 +29,7 @@ logger = get_task_logger(__name__)
|
||||
@after_app_ready_start
|
||||
@after_app_shutdown_clean_periodic
|
||||
def delete_terminal_status_period():
|
||||
yesterday = timezone.now() - datetime.timedelta(days=1)
|
||||
yesterday = timezone.now() - datetime.timedelta(days=7)
|
||||
Status.objects.filter(date_created__lt=yesterday).delete()
|
||||
|
||||
|
||||
|
||||
@@ -35,7 +35,6 @@ urlpatterns = [
|
||||
path('command-storages/<uuid:pk>/test-connective/', api.CommandStorageTestConnectiveApi.as_view(), name='command-storage-test-connective'),
|
||||
# components
|
||||
path('components/metrics/', api.ComponentsMetricsAPIView.as_view(), name='components-metrics'),
|
||||
path('components/state/', api.ComponentsStateAPIView.as_view(), name='components-state'),
|
||||
# v2: get session's replay
|
||||
# path('v2/sessions/<uuid:pk>/replay/',
|
||||
# api.SessionReplayV2ViewSet.as_view({'get': 'retrieve'}),
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
#
|
||||
import os
|
||||
from itertools import groupby
|
||||
|
||||
from django.conf import settings
|
||||
from django.core.files.storage import default_storage
|
||||
@@ -10,9 +11,7 @@ import jms_storage
|
||||
|
||||
from common.tasks import send_mail_async
|
||||
from common.utils import get_logger, reverse
|
||||
from settings.models import Setting
|
||||
from . import const
|
||||
|
||||
from .models import ReplayStorage, Session, Command
|
||||
|
||||
logger = get_logger(__name__)
|
||||
@@ -141,23 +140,73 @@ def send_command_execution_alert_mail(command):
|
||||
send_mail_async.delay(subject, message, recipient_list, html_message=message)
|
||||
|
||||
|
||||
class ComponentsMetricsUtil(object):
|
||||
|
||||
class ComputeStatUtil:
|
||||
# system status
|
||||
@staticmethod
|
||||
def get_components(tp=None):
|
||||
def _common_compute_system_status(value, thresholds):
|
||||
if thresholds[0] <= value <= thresholds[1]:
|
||||
return const.ComponentStatusChoices.normal.value
|
||||
elif thresholds[1] < value <= thresholds[2]:
|
||||
return const.ComponentStatusChoices.high.value
|
||||
else:
|
||||
return const.ComponentStatusChoices.critical.value
|
||||
|
||||
@classmethod
|
||||
def _compute_system_stat_status(cls, stat):
|
||||
system_stat_thresholds_mapper = {
|
||||
'cpu_load': [0, 5, 20],
|
||||
'memory_used': [0, 85, 95],
|
||||
'disk_used': [0, 80, 99]
|
||||
}
|
||||
system_status = {}
|
||||
for stat_key, thresholds in system_stat_thresholds_mapper.items():
|
||||
stat_value = getattr(stat, stat_key)
|
||||
if stat_value is None:
|
||||
msg = 'stat: {}, stat_key: {}, stat_value: {}'
|
||||
logger.debug(msg.format(stat, stat_key, stat_value))
|
||||
stat_value = 0
|
||||
status = cls._common_compute_system_status(stat_value, thresholds)
|
||||
system_status[stat_key] = status
|
||||
return system_status
|
||||
|
||||
@classmethod
|
||||
def compute_component_status(cls, stat):
|
||||
if not stat:
|
||||
return const.ComponentStatusChoices.offline
|
||||
system_status_values = cls._compute_system_stat_status(stat).values()
|
||||
if const.ComponentStatusChoices.critical in system_status_values:
|
||||
return const.ComponentStatusChoices.critical
|
||||
elif const.ComponentStatusChoices.high in system_status_values:
|
||||
return const.ComponentStatusChoices.high
|
||||
else:
|
||||
return const.ComponentStatusChoices.normal
|
||||
|
||||
|
||||
class TypedComponentsStatusMetricsUtil(object):
|
||||
def __init__(self):
|
||||
self.components = []
|
||||
self.grouped_components = []
|
||||
self.get_components()
|
||||
|
||||
def get_components(self):
|
||||
from .models import Terminal
|
||||
components = Terminal.objects.filter(is_deleted=False).order_by('type')
|
||||
if tp:
|
||||
components = components.filter(type=tp)
|
||||
return components
|
||||
grouped_components = groupby(components, lambda c: c.type)
|
||||
grouped_components = [(i[0], list(i[1])) for i in grouped_components]
|
||||
self.grouped_components = grouped_components
|
||||
self.components = components
|
||||
|
||||
def get_metrics(self, tp=None):
|
||||
components = self.get_components(tp)
|
||||
total_count = normal_count = high_count = critical_count = offline_count = \
|
||||
session_active_total = 0
|
||||
for component in components:
|
||||
total_count += 1
|
||||
if component.is_alive:
|
||||
def get_metrics(self):
|
||||
metrics = []
|
||||
for _tp, components in self.grouped_components:
|
||||
normal_count = high_count = critical_count = 0
|
||||
total_count = offline_count = session_online_total = 0
|
||||
|
||||
for component in components:
|
||||
total_count += 1
|
||||
if not component.is_alive:
|
||||
offline_count += 1
|
||||
continue
|
||||
if component.is_normal:
|
||||
normal_count += 1
|
||||
elif component.is_high:
|
||||
@@ -165,20 +214,23 @@ class ComponentsMetricsUtil(object):
|
||||
else:
|
||||
# critical
|
||||
critical_count += 1
|
||||
session_active_total += component.state.get('session_active_count', 0)
|
||||
else:
|
||||
offline_count += 1
|
||||
return {
|
||||
'total': total_count,
|
||||
'normal': normal_count,
|
||||
'high': high_count,
|
||||
'critical': critical_count,
|
||||
'offline': offline_count,
|
||||
'session_active': session_active_total
|
||||
}
|
||||
session_online_total += component.get_online_session_count()
|
||||
metrics.append({
|
||||
'total': total_count,
|
||||
'normal': normal_count,
|
||||
'high': high_count,
|
||||
'critical': critical_count,
|
||||
'offline': offline_count,
|
||||
'session_active': session_online_total,
|
||||
'type': _tp,
|
||||
})
|
||||
return metrics
|
||||
|
||||
|
||||
class ComponentsPrometheusMetricsUtil(ComponentsMetricsUtil):
|
||||
class ComponentsPrometheusMetricsUtil(TypedComponentsStatusMetricsUtil):
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.metrics = self.get_metrics()
|
||||
|
||||
@staticmethod
|
||||
def convert_status_metrics(metrics):
|
||||
@@ -190,50 +242,74 @@ class ComponentsPrometheusMetricsUtil(ComponentsMetricsUtil):
|
||||
'offline': metrics['offline']
|
||||
}
|
||||
|
||||
def get_prometheus_metrics_text(self):
|
||||
def get_component_status_metrics(self):
|
||||
prometheus_metrics = list()
|
||||
|
||||
# 各组件状态个数汇总
|
||||
prometheus_metrics.append('# JumpServer 各组件状态个数汇总')
|
||||
status_metric_text = 'jumpserver_components_status_total{component_type="%s", status="%s"} %s'
|
||||
for tp in const.TerminalTypeChoices.types():
|
||||
for metric in self.metrics:
|
||||
tp = metric['type']
|
||||
prometheus_metrics.append(f'## 组件: {tp}')
|
||||
metrics_tp = self.get_metrics(tp)
|
||||
status_metrics = self.convert_status_metrics(metrics_tp)
|
||||
status_metrics = self.convert_status_metrics(metric)
|
||||
for status, value in status_metrics.items():
|
||||
metric_text = status_metric_text % (tp, status, value)
|
||||
prometheus_metrics.append(metric_text)
|
||||
return prometheus_metrics
|
||||
|
||||
prometheus_metrics.append('\n')
|
||||
|
||||
def get_component_session_metrics(self):
|
||||
prometheus_metrics = list()
|
||||
# 各组件在线会话数汇总
|
||||
prometheus_metrics.append('# JumpServer 各组件在线会话数汇总')
|
||||
session_active_metric_text = 'jumpserver_components_session_active_total{component_type="%s"} %s'
|
||||
for tp in const.TerminalTypeChoices.types():
|
||||
|
||||
for metric in self.metrics:
|
||||
tp = metric['type']
|
||||
prometheus_metrics.append(f'## 组件: {tp}')
|
||||
metrics_tp = self.get_metrics(tp)
|
||||
metric_text = session_active_metric_text % (tp, metrics_tp['session_active'])
|
||||
metric_text = session_active_metric_text % (tp, metric['session_active'])
|
||||
prometheus_metrics.append(metric_text)
|
||||
return prometheus_metrics
|
||||
|
||||
prometheus_metrics.append('\n')
|
||||
|
||||
def get_component_stat_metrics(self):
|
||||
prometheus_metrics = list()
|
||||
# 各组件节点指标
|
||||
prometheus_metrics.append('# JumpServer 各组件一些指标')
|
||||
state_metric_text = 'jumpserver_components_%s{component_type="%s", component="%s"} %s'
|
||||
states = [
|
||||
stats_key = [
|
||||
'cpu_load', 'memory_used', 'disk_used', 'session_online'
|
||||
]
|
||||
old_stats_key = [
|
||||
'system_cpu_load_1', 'system_memory_used_percent',
|
||||
'system_disk_used_percent', 'session_active_count'
|
||||
]
|
||||
for state in states:
|
||||
prometheus_metrics.append(f'## 指标: {state}')
|
||||
components = self.get_components()
|
||||
for component in components:
|
||||
old_stats_key_mapper = dict(zip(stats_key, old_stats_key))
|
||||
|
||||
for stat_key in stats_key:
|
||||
prometheus_metrics.append(f'## 指标: {stat_key}')
|
||||
for component in self.components:
|
||||
if not component.is_alive:
|
||||
continue
|
||||
component_stat = component.latest_stat
|
||||
if not component_stat:
|
||||
continue
|
||||
metric_text = state_metric_text % (
|
||||
state, component.type, component.name, component.state.get(state)
|
||||
stat_key, component.type, component.name, getattr(component_stat, stat_key)
|
||||
)
|
||||
prometheus_metrics.append(metric_text)
|
||||
old_stat_key = old_stats_key_mapper.get(stat_key)
|
||||
old_metric_text = state_metric_text % (
|
||||
old_stat_key, component.type, component.name, getattr(component_stat, stat_key)
|
||||
)
|
||||
prometheus_metrics.append(old_metric_text)
|
||||
return prometheus_metrics
|
||||
|
||||
def get_prometheus_metrics_text(self):
|
||||
prometheus_metrics = list()
|
||||
for method in [
|
||||
self.get_component_status_metrics,
|
||||
self.get_component_session_metrics,
|
||||
self.get_component_stat_metrics
|
||||
]:
|
||||
prometheus_metrics.extend(method())
|
||||
prometheus_metrics.append('\n')
|
||||
prometheus_metrics_text = '\n'.join(prometheus_metrics)
|
||||
return prometheus_metrics_text
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# ~*~ coding: utf-8 ~*~
|
||||
from django.core.cache import cache
|
||||
from collections import defaultdict
|
||||
|
||||
from django.utils.translation import ugettext as _
|
||||
from rest_framework.decorators import action
|
||||
from django.conf import settings
|
||||
from rest_framework import generics
|
||||
from rest_framework.response import Response
|
||||
from rest_framework_bulk import BulkModelViewSet
|
||||
@@ -16,7 +16,7 @@ from common.mixins import CommonApiMixin
|
||||
from common.utils import get_logger
|
||||
from orgs.utils import current_org
|
||||
from orgs.models import ROLE as ORG_ROLE, OrganizationMember
|
||||
from users.utils import send_reset_mfa_mail
|
||||
from users.utils import send_reset_mfa_mail, LoginBlockUtil, MFABlockUtils
|
||||
from .. import serializers
|
||||
from ..serializers import UserSerializer, UserRetrieveSerializer, MiniUserSerializer, InviteSerializer
|
||||
from .mixins import UserQuerysetMixin
|
||||
@@ -155,10 +155,17 @@ class UserViewSet(CommonApiMixin, UserQuerysetMixin, BulkModelViewSet):
|
||||
serializer = serializer_cls(data=data, many=True)
|
||||
serializer.is_valid(raise_exception=True)
|
||||
validated_data = serializer.validated_data
|
||||
|
||||
users_by_role = defaultdict(list)
|
||||
for i in validated_data:
|
||||
i['org_id'] = current_org.org_id()
|
||||
relations = [OrganizationMember(**i) for i in validated_data]
|
||||
OrganizationMember.objects.bulk_create(relations, ignore_conflicts=True)
|
||||
users_by_role[i['role']].append(i['user'])
|
||||
|
||||
OrganizationMember.objects.add_users_by_role(
|
||||
current_org,
|
||||
users=users_by_role[ORG_ROLE.USER],
|
||||
admins=users_by_role[ORG_ROLE.ADMIN],
|
||||
auditors=users_by_role[ORG_ROLE.AUDITOR]
|
||||
)
|
||||
return Response(serializer.data, status=201)
|
||||
|
||||
@action(methods=['post'], detail=True, permission_classes=(IsOrgAdmin,))
|
||||
@@ -190,16 +197,12 @@ class UserChangePasswordApi(UserQuerysetMixin, generics.RetrieveUpdateAPIView):
|
||||
class UserUnblockPKApi(UserQuerysetMixin, generics.UpdateAPIView):
|
||||
permission_classes = (IsOrgAdmin,)
|
||||
serializer_class = serializers.UserSerializer
|
||||
key_prefix_limit = "_LOGIN_LIMIT_{}_{}"
|
||||
key_prefix_block = "_LOGIN_BLOCK_{}"
|
||||
|
||||
def perform_update(self, serializer):
|
||||
user = self.get_object()
|
||||
username = user.username if user else ''
|
||||
key_limit = self.key_prefix_limit.format(username, '*')
|
||||
key_block = self.key_prefix_block.format(username)
|
||||
cache.delete_pattern(key_limit)
|
||||
cache.delete(key_block)
|
||||
LoginBlockUtil.unblock_user(username)
|
||||
MFABlockUtils.unblock_user(username)
|
||||
|
||||
|
||||
class UserResetOTPApi(UserQuerysetMixin, generics.RetrieveAPIView):
|
||||
|
||||
@@ -667,12 +667,20 @@ class User(AuthMixin, TokenMixin, RoleMixin, MFAMixin, AbstractUser):
|
||||
else:
|
||||
return user_default
|
||||
|
||||
def unblock_login(self):
|
||||
from users.utils import LoginBlockUtil, MFABlockUtils
|
||||
LoginBlockUtil.unblock_user(self.username)
|
||||
MFABlockUtils.unblock_user(self.username)
|
||||
|
||||
@property
|
||||
def login_blocked(self):
|
||||
key_prefix_block = "_LOGIN_BLOCK_{}"
|
||||
key_block = key_prefix_block.format(self.username)
|
||||
blocked = bool(cache.get(key_block))
|
||||
return blocked
|
||||
from users.utils import LoginBlockUtil, MFABlockUtils
|
||||
if LoginBlockUtil.is_user_block(self.username):
|
||||
return True
|
||||
if MFABlockUtils.is_user_block(self.username):
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
def delete(self, using=None, keep_parents=False):
|
||||
if self.pk == 1 or self.username == 'admin':
|
||||
|
||||
@@ -322,50 +322,80 @@ def check_password_rules(password):
|
||||
return bool(match_obj)
|
||||
|
||||
|
||||
key_prefix_limit = "_LOGIN_LIMIT_{}_{}"
|
||||
key_prefix_block = "_LOGIN_BLOCK_{}"
|
||||
class BlockUtil:
|
||||
BLOCK_KEY_TMPL: str
|
||||
|
||||
def __init__(self, username):
|
||||
self.block_key = self.BLOCK_KEY_TMPL.format(username)
|
||||
self.key_ttl = int(settings.SECURITY_LOGIN_LIMIT_TIME) * 60
|
||||
|
||||
def block(self):
|
||||
cache.set(self.block_key, True, self.key_ttl)
|
||||
|
||||
def is_block(self):
|
||||
return bool(cache.get(self.block_key))
|
||||
|
||||
|
||||
# def increase_login_failed_count(key_limit, key_block):
|
||||
def increase_login_failed_count(username, ip):
|
||||
key_limit = key_prefix_limit.format(username, ip)
|
||||
count = cache.get(key_limit)
|
||||
count = count + 1 if count else 1
|
||||
class BlockUtilBase:
|
||||
LIMIT_KEY_TMPL: str
|
||||
BLOCK_KEY_TMPL: str
|
||||
|
||||
limit_time = settings.SECURITY_LOGIN_LIMIT_TIME
|
||||
cache.set(key_limit, count, int(limit_time)*60)
|
||||
def __init__(self, username, ip):
|
||||
self.username = username
|
||||
self.ip = ip
|
||||
self.limit_key = self.LIMIT_KEY_TMPL.format(username, ip)
|
||||
self.block_key = self.BLOCK_KEY_TMPL.format(username)
|
||||
self.key_ttl = int(settings.SECURITY_LOGIN_LIMIT_TIME) * 60
|
||||
|
||||
def get_remainder_times(self):
|
||||
times_up = settings.SECURITY_LOGIN_LIMIT_COUNT
|
||||
times_failed = self.get_failed_count()
|
||||
times_remainder = int(times_up) - int(times_failed)
|
||||
return times_remainder
|
||||
|
||||
def incr_failed_count(self):
|
||||
limit_key = self.limit_key
|
||||
count = cache.get(limit_key, 0)
|
||||
count += 1
|
||||
cache.set(limit_key, count, self.key_ttl)
|
||||
|
||||
limit_count = settings.SECURITY_LOGIN_LIMIT_COUNT
|
||||
if count >= limit_count:
|
||||
cache.set(self.block_key, True, self.key_ttl)
|
||||
|
||||
def get_failed_count(self):
|
||||
count = cache.get(self.limit_key, 0)
|
||||
return count
|
||||
|
||||
def clean_failed_count(self):
|
||||
cache.delete(self.limit_key)
|
||||
cache.delete(self.block_key)
|
||||
|
||||
@classmethod
|
||||
def unblock_user(cls, username):
|
||||
key_limit = cls.LIMIT_KEY_TMPL.format(username, '*')
|
||||
key_block = cls.BLOCK_KEY_TMPL.format(username)
|
||||
# Redis 尽量不要用通配
|
||||
cache.delete_pattern(key_limit)
|
||||
cache.delete(key_block)
|
||||
|
||||
@classmethod
|
||||
def is_user_block(cls, username):
|
||||
block_key = cls.BLOCK_KEY_TMPL.format(username)
|
||||
return bool(cache.get(block_key))
|
||||
|
||||
def is_block(self):
|
||||
return bool(cache.get(self.block_key))
|
||||
|
||||
|
||||
def get_login_failed_count(username, ip):
|
||||
key_limit = key_prefix_limit.format(username, ip)
|
||||
count = cache.get(key_limit, 0)
|
||||
return count
|
||||
class LoginBlockUtil(BlockUtilBase):
|
||||
LIMIT_KEY_TMPL = "_LOGIN_LIMIT_{}_{}"
|
||||
BLOCK_KEY_TMPL = "_LOGIN_BLOCK_{}"
|
||||
|
||||
|
||||
def clean_failed_count(username, ip):
|
||||
key_limit = key_prefix_limit.format(username, ip)
|
||||
key_block = key_prefix_block.format(username)
|
||||
cache.delete(key_limit)
|
||||
cache.delete(key_block)
|
||||
|
||||
|
||||
def is_block_login(username, ip):
|
||||
count = get_login_failed_count(username, ip)
|
||||
key_block = key_prefix_block.format(username)
|
||||
|
||||
limit_count = settings.SECURITY_LOGIN_LIMIT_COUNT
|
||||
limit_time = settings.SECURITY_LOGIN_LIMIT_TIME
|
||||
|
||||
if count >= limit_count:
|
||||
cache.set(key_block, 1, int(limit_time)*60)
|
||||
if count and count >= limit_count:
|
||||
return True
|
||||
|
||||
|
||||
def is_need_unblock(key_block):
|
||||
if not cache.get(key_block):
|
||||
return False
|
||||
return True
|
||||
class MFABlockUtils(BlockUtilBase):
|
||||
LIMIT_KEY_TMPL = "_MFA_LIMIT_{}_{}"
|
||||
BLOCK_KEY_TMPL = "_MFA_BLOCK_{}"
|
||||
|
||||
|
||||
def construct_user_email(username, email):
|
||||
|
||||
@@ -11,9 +11,10 @@ from django.contrib.auth import logout as auth_logout
|
||||
|
||||
from common.utils import get_logger
|
||||
from common.permissions import (
|
||||
PermissionsMixin, IsValidUser,
|
||||
IsValidUser,
|
||||
UserCanUpdatePassword
|
||||
)
|
||||
from common.mixins.views import PermissionsMixin
|
||||
from ... import forms
|
||||
from ...models import User
|
||||
from ...utils import (
|
||||
|
||||
@@ -8,9 +8,10 @@ from django.views.generic.edit import UpdateView
|
||||
|
||||
from common.utils import get_logger, ssh_key_gen
|
||||
from common.permissions import (
|
||||
PermissionsMixin, IsValidUser,
|
||||
IsValidUser,
|
||||
UserCanUpdateSSHKey,
|
||||
)
|
||||
from common.mixins.views import PermissionsMixin
|
||||
from ... import forms
|
||||
from ...models import User
|
||||
|
||||
|
||||
@@ -13,7 +13,8 @@ from formtools.wizard.views import SessionWizardView
|
||||
from django.views.generic import FormView
|
||||
|
||||
from common.utils import get_object_or_none
|
||||
from common.permissions import PermissionsMixin, IsValidUser
|
||||
from common.permissions import IsValidUser
|
||||
from common.mixins.views import PermissionsMixin
|
||||
from ...models import User
|
||||
from ...utils import (
|
||||
send_reset_password_mail, get_password_check_rules, check_password_rules,
|
||||
|
||||
@@ -122,7 +122,7 @@ REDIS_PORT: 6379
|
||||
# USER_LOGIN_SINGLE_MACHINE_ENABLED: False
|
||||
#
|
||||
# 启用定时任务
|
||||
# PERIOD_TASK_ENABLE: True
|
||||
# PERIOD_TASK_ENABLED: True
|
||||
#
|
||||
# 启用二次复合认证配置
|
||||
# LOGIN_CONFIRM_ENABLE: False
|
||||
|
||||
9
jms
9
jms
@@ -97,6 +97,14 @@ def check_migrations():
|
||||
# sys.exit(1)
|
||||
|
||||
|
||||
def expire_caches():
|
||||
apps_dir = os.path.join(BASE_DIR, 'apps')
|
||||
code = subprocess.call("python manage.py expire_caches", shell=True, cwd=apps_dir)
|
||||
|
||||
if code == 1:
|
||||
return
|
||||
|
||||
|
||||
def perform_db_migrate():
|
||||
logging.info("Check database structure change ...")
|
||||
os.chdir(os.path.join(BASE_DIR, 'apps'))
|
||||
@@ -116,6 +124,7 @@ def prepare():
|
||||
check_database_connection()
|
||||
check_migrations()
|
||||
upgrade_db()
|
||||
expire_caches()
|
||||
|
||||
|
||||
def check_pid(pid):
|
||||
|
||||
@@ -12,9 +12,9 @@ chardet==3.0.4
|
||||
configparser==3.5.0
|
||||
coreapi==2.3.3
|
||||
coreschema==0.0.4
|
||||
cryptography==3.2
|
||||
cryptography==3.3.2
|
||||
decorator==4.1.2
|
||||
Django==3.1
|
||||
Django==3.1.6
|
||||
django-auth-ldap==2.2.0
|
||||
django-bootstrap3==14.2.0
|
||||
django-celery-beat==2.0
|
||||
@@ -39,7 +39,7 @@ gunicorn==19.9.0
|
||||
idna==2.6
|
||||
itsdangerous==0.24
|
||||
itypes==1.1.0
|
||||
Jinja2==2.10.1
|
||||
Jinja2==2.11.3
|
||||
jmespath==0.9.3
|
||||
kombu==4.6.8
|
||||
ldap3==2.4
|
||||
@@ -49,7 +49,7 @@ olefile==0.44
|
||||
openapi-codec==1.3.2
|
||||
paramiko==2.7.2
|
||||
passlib==1.7.1
|
||||
Pillow==7.1.0
|
||||
Pillow==8.1.1
|
||||
pyasn1==0.4.8
|
||||
pycparser==2.19
|
||||
pycryptodome==3.10.1
|
||||
|
||||
Reference in New Issue
Block a user