Sorting in nondescending order:

Given an array A[0..n-1], reorder A 
such that A[0] <= A[1] <= A[2] <= ... <=A[n-1].


An invariant is a property that is true at 
every step of an algorithm
(at the beginning, after every recursive call/
iteration of a loop, at the end)

Bubble sort:

Main idea: go through the array, swapping adjacent elements
if they are out of order, until the largest element goes to the
end.  Repeat this process until no swaps are made or 
this process has been done n-1 times, where n=arr.length

invariant:

After p passes through the array, the last p elements 
are in the proper order. 


public static void bubblesort(int[] arr) {

	boolean isSorted = false;
	for(int i=0; i<arr.length-1 && !isSorted; i++){
		isSorted=true;
		for(int j=0; j<arr.length-i-1; j++){
			if(arr[j]>arr[j+1]){
				swap(arr,j,j+1);
				isSorted=false;
			}
		}
	}
} 

Best case: Array is already sorted 
running time: O(n) time (I have to go to the array once).

Worst case: Array is sorted in nonascending order.

running time: n-1 + n-2 + n-3 + ... + 2 + 1
 = 1 + 2 + 3 + ... + n-1
 = n(n-1)/2 = O(n^2).



Insertion sort: 

main idea: You split the array into 2 parts, a sorted subarray
and an unsorted subarray, where A[0..i-1] is sorted 
and A[i..n-1] is unsorted. 
At each step, look at A[i] and put it into the sorted part 
of the array in its proper place by shifting down all 
elements > A[i].  
After you do this for A[1], A[2], ..., A[n-1], 
the array is sorted. 

invariant:
After pass i, A[0..i] is already sorted.    

public static void insertionSort(int[] arr) {
	for(int i=1; i<arr.length; i++) {

		int element = arr[i]; //thing needs to be inserted
		int loc = i-1;
	
		while(loc>=0 && element < arr[loc]) {
			arr[loc+1] = arr[loc];
			loc--;
		}
		arr[loc+1] = element;
	}
}

Best case: Every element in the array is a constant 
number of spaces away from where it belongs in the sorted
order. (special case: if everything is 0 spaces away
= its sorted) 

running time: O(n) 

Worst case: The array is sorted in nonascending order 

running time: 1 + 2 + 3 + ... + n = n(n+1)/2 = O(n^2).

Although bubble sort and insertion sort both take O(n^2)
time, the constant factor behind the O-notation of 
insertion sort is much smaller than that of bubble sort 
in almost all cases.  

--------------------------------------------

Mergesort:

Merging:

Suppose I had 2 sorted arrays and I wanted to create one
big sorted array out of them.  How can I do it? 

1. Create a third array whose size is the combined size
of the 2 smaller arrays
2. Compare the smallest number in each array, and copy 
it into the third array. Repeat this until you have no
numbers left. 

private static void merge(int[] arr, int start, int middle,
			int end)
{
	//arr[start..middle-1] is already sorted
	//arr[middle..end-1] is already sorted

	//goal is to merge these 2 subarrays 
	// and put the result into arr[start..end-1]

        int[] result = new int[end-start];
     
        int index1 = start;
        int index2 = middle;
        int indexR = 0;

	while(index1 < middle && index2 < end) {
		if(arr[index1]< arr[index2])
			result[indexR] = arr[index1];
			index1++;
		}else{
			result[indexR] = arr[index2];
			index2++;
		}
		indexR++;
	}
  
        while(index1 < middle) {
		result[indexR] = arr[index1];
		index1++;
		indexR++;
	}
	while(index2 < end) {
		result[indexR] = arr[index2];
		index2++;
		indexR++;	
	}

	for(int i=0; i<result.length; i++)
		arr[start+i] = result[i];
}

Running time for merging: 
Let n = end-start = the combined size of the 2 subarrays

worse case running time = O(n) because I copy a total of n
elements, and only consider the smallest element 
that remains once. 

Mergesort:

1. use mergesort to recursively sort the first half 
of the array and the second half of the array.
2. merge them together. 


public static void mergesort(int[] arr) {
	mergesort(arr, 0, arr.length);
}

private static void mergesort(int[] arr, int start, int end) {

	if(end-start<=1)
		return;
	
	int middle = (end+start)/2;
	mergesort(arr, start, middle);
	mergesort(arr, middle, end);
	merge(arr, start, middle, end);
} 

Def: An algorithm is said to be inplace if it uses
only a constant amount of extra memory in order to run. 

E.g. Bubblesort is inplace. So is insertionSort. 
     Mergesort is NOT inplace, because the merge requires
     O(n) extra space to run. 
     (problem: an array of size 4GB, mergesort would require
      ~ 8 GB total to run, which might exceed your RAM)
 

problem: Suppose we had k sorted arrays to merge, each of 
size n.  How would we do that? 

1. Merge the first 2 arrays, then merge that with the third,
   merge that with the 4th, etc. 

   2n + 3n + 4n + .... +kn = O((k^2)n)

2. Find the minimum of the k arrays, pick that element,
   increment the index of that array. Repeat until all elements
   are exhausted. We have kn elements, and for each one
   we examine the kn minima, so we get time O((k^2)n). 

3. Take all the arrays, combine them all and sort.
    O(kn log kn). if k = O(n) this is like O(kn log n).
    
    (log (n^3) = 3 log n)

4.  Modify idea 2, but instead of finding the minimum of 
    the k arrays directly, put the in a minimum priority 
    queue (implemented using a heap) and find the minimum
    that way.  Then when we extract the minimum, put the 
    next candidate into the heap. 
    Each time we modify the heap, it takes O(log k) time
    and we do this kn times, so we get time O(kn log k)

    if n = O(k^a), then log n = log(k^a) = a log k 
        
---------------------------------------------------
Quicksort: Sir C.A.R. Hoare (early 1960's) 

Main Idea: Suppose we took the array and chose an element within
called the pivot.  We put everything <=pivot on the left
and everything >=pivot on the right.  Once we do that,
if we sort the left and right, the array will be sorted.

The way to prepare the array is by partitioning. 

Idea of the partition procedure I will present is as follows:

The array[start..end] will be split into 3 parts:

Every element in array[start..pivotIndex] is <= pivot 
Every element in array[pivotIndex+1..i-1] >= pivot
Every element in array[i..end] is unknown. 

public static int partition(int[] arr, int start, int end)
{
       Random r = new Random();
       int randomIndex = r.nextInt(end-start);
       swap(arr, start, randomIndex);
       int pivot = arr[start];
       int boundary = start; //at the beginning, there is one
			     element <=pivot (pivot itself)

        for(int i=start+1; i<end; i++)
		if(arr[i] < pivot) {
			boundary++;
			swap(arr, i, boundary);
		}
	swap(arr, start, boundary); //moves the pivot into place
        return boundary;
}

public void swap(int[] arr, int i, int j){
	int temp = arr[i];
	arr[i] = arr[j];
	arr[j] = temp;
}

public static void quicksort(int[] arr) {
	quicksort(arr, 0, arr.length);
}

private static void quicksort(int[] arr, int start, int end) {
	if(end-start<=1)
		return;

	int boundary = partition(arr, start, end);
	quicksort(arr, start, boundary);
	quicksort(arr, boundary+1, end);
}

best case: The pivot always splits the array in some
predictable fractional way (e.g. in half). 
Running time is O(n log n) because it mirrors the recurrence
for merge sort. 

worst case: If the array is sorted or reverse sorted. 
The partition doesn't split the array into 2 parts.
You get one empty subarray and one subarray of size n-1. 
Running time: n + n-1 + n-2 + n-3 + ... + 1 = O(n^2). 

If this is O(n^2) worst case, it's not so quick!

On average (in expectation), if you choose a random pivot,
the likelihood of the worst case is 
happening -> 0 as n->infinity. The running time is almost
always O(n log n).