Thread synchronization: Mechanism to ensures that two or more concurrent threads do not simultaneously execute some particular program segment known as critical section
Critical section refers to the parts of the program where the shared resource is accessed.
Issues in Synchronization
Race Condition: Occurring of a condition when two or more threads can access shared data and then try to change its value at the same time. Due to this, the values of variables may be unpredictable and vary depending on the timings of context switches of the processes.
Python Code: Without Synchronization
import threading
x = 0
def increment_global():
global x
x += 1
def taskofThread():
for _ in range(50000):
increment_global()
def main():
global x
x = 0
t1 = threading.Thread(target= taskofThread)
t2 = threading.Thread(target= taskofThread)
t1.start()
t2.start()
t1.join()
t2.join()
if __name__ == "__main__":
for i in range(5):
main()
print("x = {1} after Iteration {0}".format(i,x))
OUTPUT
x = 100000 after Iteration 0
x = 63883 after Iteration 1
x = 82552 after Iteration 2
x = 100000 after Iteration 3
x = 68994 after Iteration 4
Python Code: Locking Mechanism and Synchronization
Virtualization is the use of virtual machine that can virtualize the greater part of the equipment assets, including processors, memory, and network availability. With the virtualization, physical equipment assets can be shared by at least one virtual machines. According to the necessities from Popek and Goldberg, there are three perspectives to fulfill the virtualization. To start with, the virtualization ought to give an identical domain to run a program contrasted with a local framework.
To comprehend virtualization, hypervisor ought to be addressed first. Hypervisor empowers correspondence amongst equipment and a virtual machine so that the virtualization fulfills with this deliberation layer (hypervisor). Hypervisor is originally called virtual machine screen (VMM) from these two terms (Hypervisor and VMM) are regularly regarded as equivalent words, yet according to the distinction from Agesen et al, a virtual machine screen (VMM) is a product that manages CPU, memory, I/O data exchange, interrupt, and the instruction set on a given virtualized environment.
A hypervisor may allude to an operating framework (OS) with the VMM. Commonly, a hypervisor can be partitioned into Type 1 and Type 2 hypervisor in view of the diverse level of execution. Sort 1 is sitting on equipment and the correspondence amongst equipment and virtual machine is immediate. The host operating framework is not required in Type 1 hypervisor since it runs straightforwardly on a physical machine. Because of this reason, it is now and again called an ‘exposed metal hypervisor’.
VMware vSphere/ESXi, Microsoft Windows Server 2012 Hyper-V, Citrix XenServer, Red Hat Enterprise Virtualization (RHEV) and open-source Kernel-based Virtual Machine (KVM) are distinguished in this class. Sort 2 hypervisor is on the operating framework to manage virtual machine effectively with the support of equipment arrangement from operating framework. The additional layer amongst equipment and virtual machine in the sort 2 hypervisors causes inefficiency contrasted with the sort 1 hypervisor. Virtual Box and VMware Workstation are in this class.
The terms of Host or Guest machine (or domain) are utilized as a part of the hypervisor to depict diverse parts. Host machine (domain) contains a hypervisor to manage virtual machines, and Guest machine (domain) implies each virtual machine sitting on a hosted machine in a protected and confined environment with its own consistent domain.
With these separated parts, the hypervisor can offer asset limits to different virtual machines on the same physical machine. In other words, the hypervisor is a product layer that makes a virtual domain with virtualized CPU, memory and I/O (stockpiling and network) gadgets by abstracting without end the underlying physical equipment. Virtual machine (VM) commonly alludes to an epitomized substance including the operating framework and the applications running in it also.
TYPES OF VIRTUALIZATION
· Full virtualization — Full virtualization gives virtualization without modifying guest operating framework. In x86 engineering, dealing with benefits instructions is a key component for virtualizing equipment. VMware offers binary interpretation of operating framework asks for so that virtualizing favored instructions can be finished without backings from either equipment or operating framework. There are Microsoft Virtual Server and VMware ESXi using this system.
· Para-virtualization- Xen gather initially created paravirt-operations (later called by paravirtualization) to bolster elite and effective asset segregation with slight adjustments to the guest operating framework. Xen saw that full virtualization upheld guest domains without an adjustment in the operating framework, yet there were negative influences on performance because of the utilization of shadow page tables. Paravirtualization (PV) requires the adjusted OS kernel with framework calls to manage advantaged instructions. Xen registers guest OS page tables specifically with the MMU with a read-just access to keep away from the overhead and intricacy regarding the updating shadow page tables in full virtualization. With the interface between a virtual machine and a hypervisor, paravirtualization accomplishes superior without the help from equipment expansions on x86. Paravirtualization underpins unmodified application binary interface (ABI) so that client applications don’t require any progressions. Paravirtualization is additionally called operating framework assisted virtualization in light of the familiarity with a hypervisor on guest OS. Xen, UML and VMware bolster paravirtualization.
·Hardware Assisted virtualization — To enhance performance of virtualization, Intel and AMD gives virtualization expansions to x86 processors. Intel Virtualization Technology (VT) and AMD Virtualization (AMD-v) are increasing velocities for favored instructions including memory management unit (mmu), coordinated I/O gadgets (iommu). With this equipment assisted virtualization innovation, altered guest OS is pointless to empower virtualization in light of the fact that VMM manages benefit instruction at a root mode which is a ring — 1 without affecting the guest OS.
Using Second Level Address Translation (SLAT), settled paging in Intel EPT (Extended Page Table) or AMD RVI (Rapid Virtualization Indexing), memory management has been improved and the overhead of translating guest physical addresses to genuine physical addresses has been decreased. Early CPUs for x86 don’t have virtualization expansions which are not included in equipment assisted virtualization.
· Operating System-level virtualization (Shared Kernel Virtualization) — Operating framework gives separated allotments to run virtual machines in a similar kernel. With a chroot operation, which is a move of a root catalog for a certain procedure with a detachment to outside registries, OS-level virtualization empowers segregation between numerous virtual machines on a mutual OS. Overhead is exceptionally restricted in this model because of the advantages of running under operating frameworks with a common kernel. Emulating gadgets or communicating with VMM is a bit much. The guest os and the host os ought to have a similar OS or kernel. Running Windows on Linux host is incompatible. There are Solaris Containers, BSD Jails, LXC, Linux vServer, and Docker.
SOFTWARE IMPLEMENTATIONS FOR VIRTUALIZATION
Bare-metal virtualization hypervisors
· Microsoft Hyper-V
· VMware ESX and ESXi
· Citrix XenServer
· Oracle VM
Hosted virtualization hypervisors
· VMware Workstation/Fusion/Player
· VMware Server
· Microsoft Virtual PC
· Oracle VM VirtualBox
· Red Hat Enterprise Virtualization
· Parallels Desktop
CONTAINERS
The approach of online programming, administrations and stages alongside commoditization of computing administrations implies that scaling has happened to paramount significance. Considerably more, new paradigms in programming advancement — lithe systems, which abbreviate the way between the engineer and the generation situations, additionally added to an increase in appropriation among ventures and new businesses alike.
These new advancement hones and another arrangement of necessities, as far as continuous integration and continuous sending meant that there was an appropriate seeding ground for thoughts like containerization. Containerization tools empower “unchanging nature” in the infrastructure: container applications that are worked ‘on the spot’ and ‘set up’ with an indistinguishable setup and same conditions from the original creators intended.
This is one of the main reasons (or maybe even the main driving power) of their fast selection and increased utilization. These tools permit application engineers to concentrate on the application instead of focusing on the infrastructure while bringing versioning to application picture dispersion. It even goes similarly as bringing ideas that “customarily” had a place with an alternate region: you can do pulls, pushes and confers on Docker pictures, ideas obtained from Source Code Management Software (like Git and Mercurial).
To get any disarray off the beaten path, Docker alludes both to an open-source extend (https://github.com/docker/docker) and additionally the organization behind it — Docker Inc. (once in the past dotCloud). dotCloud was a PaaS player who manufactured Docker for internal utilize. When they understood its potential, they rotated and concentrated only on developing Docker.
The significance of the venture in the more noteworthy biological system was likewise perceived by the vast majority of the industry, along these lines allowing Docker to bring 40M$ up in the latest funding round. Following the turn dotCloud (the PaaS offering of the organization) was “migrated” to Berlin-based cloudControl. Docker is only one of the accessible containerization offerings and, the same number of comparative ventures, depends on kernel highlights that (in Linux) have been accessible for over 6 years (since around 2007).
Notwithstanding, as with other innovations (see e.g. blast in cell phone deals with the introduction of iPhone) Docker has just put a client (and designer) neighborly interface around prior parts. Similar ideas showed up 2–3 years prior in Solaris OS (assemble 51 in Solaris 10), known as Solaris Containers w/Solaris Zones. While initially Docker was a wrapper for LXC (Linux Containers), these days it manages libcontainer — a brought together interface for cgroups and kernel namespaces.
DOCKER
While there is a huge amount of likenesses there are likewise no less than two tons of contrasts between the two. The most important distinction is that, in Docker, there’s no overhead introduced by the need to copy a whole OS for the virtual machine. Thusly, Docker gives better performance (as far as speed and limit) when for custom applications in a container, contrasted with virtual machines, gave that the VM host and LXC host have a similar equipment qualities.
Both containers and Virtual Machines address a similar issue — disconnection and control of parts of an application — however this is accomplished in various path as containers surrender a portion of the separation for a more effective use of the (host) framework assets.
Docker depends for its execution surroundings, on elements in the host’s kernel — LXC. Be that as it may, it additionally needs filesystem bolster in the purported UFS (Union File System). Docker Filesystems Multilayer Early forms of Docker depended on AuFS, yet later discharges have received a “nonpartisan” approach, with an inclination for device mapper.
This utilization of a duplicate on-compose FS is really what makes Docker look like Git (and Docker Hub like GitHub). Docker’s two main ideas are Containers and Images, where Containers are, well, containers running the application and Images are the Containers-at-Rest (i.e. spared container state). To make a relationship with class ideas in OOP, Images are class definitions, while Containers are class instances at run-time.
With everything taken into account there are only 28 commands that the tool called Docker (self-entitled “independent runtime” for Linux containers) sees, yet they wrap all the specified capacities — including the control, (for example, begin/stop a container) and the meta-management, (for example, push/draw to/from the store).
For instance, starting a container running Redis is as basic as*:
$ docker pull mydockerfile/redis
$ docker run -d — name redis -p 6379:6379 mydockerfile/redis
This will allow one to subsequently use to connect to the Redis server.
<container_host_ip>:6379
The latest version of Ubuntu Linux (14.04 LTS) comes with activated Docker support. It still needs to be installed, though that’s just an ‘apt-get install docker.io’.
KEY FEATURES OF DOCKER
Application-driven: the incremental change that Docker brings on LXC is core interest on organization of use versus sending of machines
Compactness: once distributed, any picture will yield precisely the same wherever it runs • versioning: much like Git, permits submit/push and pulls on existing pictures, verifying contrasts from one focus on the other
Computerization: there are tools that permit a machine, once running, to achieve a particular state
Sharing: using Docker Hub anybody can expand on existing machines or make accessible their pictures for others
Reusability: a picture can be ‘fork’- ed for two distinct purposes.
INSTALLING DOCKER ON WINDOWS
The initial step is to download and install the most recent Docker for Windows work from its GitHub storehouse. As should be obvious in the following figure, Docker for Windows utilizes a conventional Windows installer.
In the event that you acknowledge the installation defaults (which I exceptionally recommend you do), the product gives you the following three parts:
Boot2Docker management tool and ISO: This is the Docker administration command-line interface (CLI).
Virtual Box: We require the center of the Oracle Virtual Box VM desktop virtualization item to permit us to run the center Linux code from inside Windows.
MSYS-git UNIX tools: The Docker Hub will pull containers from open Git storehouses, so we need customer tools accessible on our framework.
Next, double tap the Boot2Docker Start Desktop symbol to begin the Boot2Docker management tool. You’ll be provoked to permit Virtual Box to bind two virtual network interfaces, and after a short time you’ll see an unequivocally Linux-y command quick, as appeared in the following screenshot.
LAUNCHING TENSORFLOW
Open up “Docker Quickstart Terminal”. It ought to bring up a MinGW-sort shell. You could likewise utilize Windows cmd.exe or Powershell.exe however they have additional arrangements that you have to do before you can run docker.exe
Dispatch the TensorFlow container. Set up port forwarding, mount substance of registry into/home.
$ docker run — it — p 8888:8888 — p 6006:6006 — v/c/Users/name/Dropbox/mypath:/home b.gcr.io/tensorflow/tensorflow
The first run through the command is run, it will Download and install TensorFlow. A short time later, this ought to bring you into a Linux VM.
The command above mounts an organizer in your Windows host machine into the container. Its desirable over do things thusly, in light of the fact that the container does not hold on your documents as a matter of course.
It as of now accompanies Jupyter installed, yet you won’t have the capacity to get to the note pads by navigating to localhost:8888 (or whatever port Jupyter begins on). You will likewise want to open port 6006 to have the capacity to utilize TensorBoard (right now, exposing a port on a live container is unrealistic).
Instead, since it is running in a VM, not just do you have to forward the port (thus the — p 8888:8888), yet the IP address you get to should be the IP address of the VM, not the Windows Machine. Thus, you have to find the address of the docker machine running the container.
Listing the docker containers.
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
444413d7235233 b.gcr.io/tensorflow/tensorflow "/bin/bash" 2 minutes ago Up 2 minutes 6006/tcp, 0.0.0.0:8888->8888/tcp MyPath
Navigate web browser to 192.168.99.100:8888 (or whatever webserver port your web app is running on) and you should be able to see your web apps.
TENSORFLOW AS A RESEARCH ENVIRONMENT
If you want to use this container for your research environment all you need to do is to clone the docker container, so that you can make changes to it.
In the Docker Quickstart terminal,
docker pull b.gcr.io/tensorflow/tensorflow-full
$ docker images
For example if you installed a python package like ipdb and want to make changes, type:
$ docker commit MyPath mypath/tensorflow
$ docker run mypath/tensorflow
Now, the container is ready for research purposes and implementation of algorithms.
This article focus on the assorted types of virtualization approaches, tools and technologies with the specific case of Docker Implementation for different applications. The concept of virtualization and its applications are quite immense and the more we dig into the matter, the more astonishing results we can fetch.
BitDegree is providing Free Certifications during lockdown in the domain of Digital Marketing. The students, researchers, practitioners, academicians and industry experts can join these certifications in free of cost https://www.bitdegree.org/tag/covid
The researchers in data science, machine learning, deep learning and related approaches work on different types of datasets. Traditionally, there is need to work with the benchmark dataset so that the validation can be done and outputs will be accepted.
Many times, the researchers collect their own datasets and then use it for implementation of algorithm.
To convert the own collected data to benchmark data, following should be implemented and the dataset should be having specific properties
The dataset should be focused towards a specific type of machine learning task
The dataset should be open without any restrictions on download by other researchers
The dataset should be having sufficient features so that training, testing and validation can be done
The dataset should be accessible by other researchers and practitioners so that they can validate the outcomes
The dataset should be having labels for identification of attributes
The dataset should be clean from mismatch and without missing values
The dataset should not be very huge is size
There should be proper documentation of the dataset with its details of attributes
With the impact of CoronaVirus (COVID-19), there is lockdown at many locations. Software Testing is one of the key domains where the Work From Home (WFH) can be done. In this segment, the software applications including mobile apps are tested and the remote professionals are paid very good by the companies.
Following are the portals from where the work can be taken and payments are received in Dollars
While open sourcing a project, a license should be taken so that the distribution, linking, modification, private use etc. can be assigned
Open source licenses help others to contribute to your work or project without seeking special individual permission to do so
Not Fully Open Source
MIT Open Courseware Material is released under CC
Four Parts of CC
Accreditation: Author must be attributed as the creator of the work. Then, work can be modified / distributed / copied / used
Sharing: The work can be modified, distributed but only under CC License.
Non-Commercial: Work can be modified, distributed but not for commercial purposes.
No Derivative Works: Can copy and distribute the licensed work, but you can’t modify it in any way or create work based on the origin [as MIT Open Courseware Material is]
Attribution CC BY
This license lets others distribute, remix, adapt, and build upon your work, even commercially, as long as they credit you for the original creation. This is the most accommodating of licenses offered. Recommended for maximum dissemination and use of licensed materials.
Attribution-ShareAlike
CC BY-SA
This license lets others remix, adapt, and build upon your work even for commercial purposes, as long as they credit you and license their new creations under the identical terms. This license is often compared to “copyleft” free and open source software licenses. All new works based on yours will carry the same license, so any derivatives will also allow commercial use. This is the license used by Wikipedia, and is recommended for materials that would benefit from incorporating content from Wikipedia and similarly licensed projects.
Attribution-NoDerivs
CC BY-ND
This license lets others reuse the work for any purpose, including commercially; however, it cannot be shared with others in adapted form, and credit must be provided to you.
Attribution-NonCommercial
CC BY-NC
This license lets others remix, adapt, and build upon your work non-commercially, and although their new works must also acknowledge you and be non-commercial, they don’t have to license their derivative works on the same terms.
Attribution-NonCommercial-ShareAlike
CC BY-NC-SA
This license lets others remix, adapt, and build upon your work non-commercially, as long as they credit you and license their new creations under the identical terms.
Attribution-NonCommercial-NoDerivs
CC BY-NC-ND
This license is the most restrictive of our six main licenses, only allowing others to download your works and share them with others as long as they credit you, but they can’t change them in any way or use them commercially
