Skip to content

Gevent refactoring - #862

Merged
VakarisZ merged 4 commits into
guardicore:developfrom
VakarisZ:gevent_refactoring
Dec 17, 2020
Merged

VakarisZ merged 4 commits into
guardicore:developfrom
VakarisZ:gevent_refactoring

Conversation

@VakarisZ

@VakarisZ VakarisZ commented Oct 12, 2020 •

Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes #858 #857

Refactored tornado WSGI container into gevent WSGI container.

PR Checklist

  • Have you added an explanation of what your changes do and why you'd like to include them?
  • Is the TravisCI build passing?

Testing Checklist

  • Added relevant unit tests?
  • Have you successfully tested your changes locally? Elaborate:

    Tested locally, on windows

@acepace

acepace commented Oct 12, 2020

Copy link
Copy Markdown
Contributor

While i take a look
Can you explain the need? I can guess but want to make sure.

@VakarisZ

Copy link
Copy Markdown
Contributor Author

@acepace take a look into the issues this fixes.

debug_message += " through proxies: %s" % ControlClient.proxies
LOG.debug(debug_message)
requests.get("https://%s/api?action=is-up" % (server,), # noqa: DUO123
requests.get(f"https://{server}/api?action=is-up", # noqa: DUO123

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What version of python are we baselined on? 3.6 or 3.7?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3.7


def get_windows_commands_to_schedule_jobs():
return f'schtasks /Create /SC monthly /TN {SCHEDULED_TASK_NAME} /TR {SCHEDULED_TASK_COMMAND}'
return f'schtasks /Create /SC monthly /F /TN {SCHEDULED_TASK_NAME} /TR {SCHEDULED_TASK_COMMAND}'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't be part of this PR but shrug

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, didn't notice other commits

Comment thread monkey/monkey_island.py Outdated
Comment on lines +1 to +2
from gevent import monkey
monkey.patch_all()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confusing name, wouldn't it be nicer to
from gevent import monkey as gevent_monkey

since our entire codebase uses monkey a lot

Comment on lines +11 to +12
from monkey_island.cc.services.reporting.report_generation_synchronisation import safe_generate_reports, \
is_report_being_generated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't be part of this PR but shrug

@@ -1,15 +1,15 @@
import logging
import threading
from gevent.lock import Semaphore

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BoundedSemaphore

@acepace

acepace commented Oct 14, 2020

Copy link
Copy Markdown
Contributor

LGTM

@codecov

codecov Bot commented Oct 16, 2020 •

Copy link
Copy Markdown

Codecov Report

Merging #862 (d0fda6b) into develop (f07826d) will decrease coverage by 0.03%.
The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop     #862      +/-   ##
===========================================
- Coverage    60.56%   60.52%   -0.04%     
===========================================
  Files          166      165       -1     
  Lines         4953     4948       -5     
===========================================
- Hits          3000     2995       -5     
  Misses        1953     1953              
Impacted Files Coverage Ξ”
infection_monkey/transport/http.py 21.15% <0.00%> (-0.38%) ⬇️
infection_monkey/control.py 21.10% <0.00%> (-0.37%) ⬇️
common/common_consts/timeouts.py

Continue to review full report at Codecov.

Legend - Click here to learn more
Ξ” = absolute <relative> (impact), ΓΈ = not affected, ? = missing data
Powered by Codecov. Last update ac71a3e...9e9518b. Read the comment docs.

@acepace

acepace commented Nov 30, 2020

Copy link
Copy Markdown
Contributor

Can you rebase and we merge?

@itaymmguardicore itaymmguardicore left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good to me. If @acepace doesn't have any further comments I'm good with it.
My only concern is the transition from single-thread to multi-thread can cause unexpected errors in different parts in code which weren't designed properly or didn't take this into consideration.

I suggest running some/all automatic blackbox tests on this branch to make sure we didn't break anything

@VakarisZ

Copy link
Copy Markdown
Contributor Author

@itaymmguardicore so let's do a proper tests before the next release and catch all errors, not only something caused by gevent. Cost of running in depth tests > info that specific bug happened due to gevent.

@VakarisZ
VakarisZ merged commit 7fb1e3f into guardicore:develop Dec 17, 2020
@VakarisZ
VakarisZ deleted the gevent_refactoring branch February 18, 2022 06:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Blocking island server

3 participants